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FOREWORD 


These  Proceedings  preserve  in  print  many  of  the  invited  addresses  and  con¬ 
tributed  papers  presented  at  the  Thirtieth  Conference  on  the  Design  of 
Experiments  in  Army  Research,  Development  and  Testing.  These  meetings  are 
sponsored  by  the  Army  Mathematics  Steering  Committee  (AMSC)  on  behalf  of  the 
Office  of  the  Chief  of  Research,  Development  and  Acquisition.  Members  of  this 
Committee  have  requested  that  the  guest  lecturers  be  effective  researchers  who 
are  in  frontier  fields  of  live  current  Interest.  They  feel  that  the  addresses 
by  the  principal  speakers  as  well  as  contributed  papers  by  Army  personnel  will 
stimulate  the  Interchange  of  ideas  among  the  scientists  attending  said  meet¬ 
ing.  Noted  below  is  a  list  of  the  Invited  speakers  together  with  the  titles 
of  their  addresses. 

Speakers  and  Affiliations  Title 

Professor  John  W.  Tukey  Limited  Randomization  as  the  Key  to  Taking 

Princeton  University  and  Advantage  of  Modern  Summaries 

Bell  Laboratories 

professor  Roy  E.  Welsch  Regression  Diagnostics 

Massachusetts  Institute  of 

Technology 

Professor  James  Bucklew  Quantization 

University  of  Wisconsin 

Professor  Ulf  Grenander  Recent  Work  In  Pattern  Theory 

Brown  University 

Dr.  Ronald  L.  Iman  Uncertainty  Analysis  and  Sensitivity 

Sandia  Laboratories  in  R*sk  Assessment 

Professor  Bernard  Harris  System  Reliability 

University  of  Wisconsln-Madlson 

Tne  U.  S.  Army  White  Sands  Missile  Range,  on  22-24  August  1980,  served  as  host 
for  the  Twenty-Sixth  Conference  on  the  Design  of  Experiments,  and  the  meeting 
was  conducted  on  the  campus  of  its  co-host,  the  New  Mexico  State  University 
(NMSU)  In  Las  Cruces,  New  Mexico.  This  proved  to  be  an  Ideal  arrangement,  and 
members  of  the  AMSC  were  pleased  to  hear  it  could  be  used  for  the  Thirtieth 
Conference.  This  meeting  took  place  on  17-18  October  1984  In  the  Physical 
Science  Laboratory  of  NMSU.  The  attendees  would  like  to  thank  Mr.  Robert  A. 
Voss,  the  Chairman  on  Local  Arrangements,  for  all  his  efforts  In  planning  and 
conducting  this  conference. 


A  couple  of  days  before  the  start  of  the  Design  Conference,  a  two-day  tutorial 
entitled  "The  Bootstrap"  was  presented  by  Professor  Robert  Tibshirani  of 
Stanford  University.  The  main  purpose  of  this  seminar  was  to  develop,  in  Army 
scientists,  an  appreciation  for  and  the  necessary  skills  needed  to  handle  this 
statistical  method. 

professor  Nozer  D.  Slngpurwal la.  Department  of  Operations  Research  at  George 
Washington  University,  was  selected  by  the  AMSC  to  receive  the  Fourth  Wilks 
Award  for  Contributions  to  Statistical  Methodologies  In  Army  Research, 
Development  and  Testing.  The  citation  for  his  award  reads: 

"For  singular  contributions  to  reliability  theory  and  life 
testing  methodologies,  for  professional  service  to  the 
statistics  community,  and  for  invaluable  assistance  In 
solving  several  Important  testing  problems  In  the  Department 
of  Defense." 

There  are  many  Individuals  and  things  that  contribute  to  the  success  of  these 
scientific  meetings  such  as  the  speakers,  invited  and  contributed  panelists, 
the  chairpersons,  members  of  the  audience,  as  well  as  the  hosts  and  their 
facilities.  Sometimes  one  overlooks  the  contributions  of  the  members  of  the 
Program  Committees,  so  this  year  the  AMSC  would  like  to  take  this  occasion  to 
express  its  appreciation  for  their  valuable  scientific  contributions  to  the 
stature  of  these  conferences.  The  names  of  the  committee  members  for  the 
Thirtieth  Conference  are: 


PROGRAM  COMMITTEE 


Carl  Bates 
Larry  Crow 
Bernard  Harris 
Robert  Launer 
J.  Richard  Moore 


Carl  Russell 
Douglas  Tang 
Malcolm  Taylor 
Jerry  Thomas 
Langhorne  Withers 
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ABSTRACT 


A  regression  is  constructed  using  prior  knowledge,  data,  models,  and  a 
fitting  (estimation)  process  of  some  form*  It  is  important  to  know  when  the 
resulting  regression  depends  heavily  on  a  small  part  of  the  prior  knowledge, 
on  a  small  part  of  the  data,  or  on  the  exact  choice  of  model  or  fitting 
process.  We  want  models  that  are  sensitive  to  the  issues  of  interest,  but 
perform  woll  (i.e.,  are  less  sensitive)  when  assumptions  are  violated.  In 
this  paper  we  present  an  overview  of  the  theory,  application,  and  computation 
of  regression  diagnostics,  especially  those  related  to  the  analysis  of 
influential  data.  . 


INTRODUCTION 


Our  basic  goal  in  this  paper  is  to  learn  if  our  regression  is  heavily 
influenced  by  small  subsets  of  data.  A  traditional  starting  point  is  to  look 
for  outliers  which  may  be  viewed  as  observations  that  appear  to  be  surprising 
to  the  investigator«  or  observations  Chat  are  not  a  realisation  from  some 
target  distribution.  It  is  essential  that  all  data  used  in  regression  models 
be  examined  for  outliers.  The  first  step  is  to  look  at  the  responee  and 
explanatory  variables  separately  to  get  a  feeling  for  outliers.  At  this  time, 
transformations  of  these  variables  might  be  considered.  A  heavily  skewed  data 
series  may  appear  to  have  lota  of  outliers.  A  logarithmic  transformation  may 
make  the  outliers  appear  much  more  like  the  rest  of  the  data.  Such  transfor¬ 
mations  are,  of  course,  tentative  and  need  to  be  considered  in  light  of  prior 
knowledge  and  subsequent  results. 

Such  a  univariate  examination  does  not  help  us  find  outliers  relative  to  a 
particular  model  and  fitting  process.  In  fact,  some  of  the  univariate 
outliers  may  not  look  so  discrepant  in  the  context  of  a  multivariate  model. 
Conversely,  and  more  commonly,  multivariate  outliers  will  arise  which  cannot 
be  seen  in  a  univariate  analysis. 

A  particularly  useful  way  to  detect  outliers  in  the  context  of  a  model  is 
to  look  for  overly  influential  subsets  of  data.  Subsets  of  data  are  regarded 


as  influential  if  their  deletion  results  in  substantial  changes  to  important 
features  of  an  analysis. 

Our  discussion  of  regression  diagnostics  starts  with  preliminary  steps 
that  are  necessary  before  the  decision  is  made  to  use  a  least-squares  linear 
regression  model.  Then  we  introduce  the  idea  of  adjusted  variables  and 
partial  regression  plots.  After  a  brief  diacussion  of  collinearity 
diagnostics,  we  define  leverage  and  several  different  kinds  of  residuals.  We 
then  go  on  to  measure  influence  and  develop  plots  to  summarise  influential 
data  diagnostics.  We  conclude  with  a  brief  treatment  of  diagnostics  for 
generalised  linear  models  and  comments  about  some  areas  of  research  in 
diagnostic  methods. 

PWT.TMTMA1T  8TXP8 

There  are  many  reasons  for  performing  a  regression  analysis.  Two  of  the 
most  common  are: 

(a)  fitting  an  equation  or  model  to  data 

(b)  attempting  to  describe  local  averages  of  y  about  values  of  x 

E(y|x)  ■  g(x). 

Both  of  these  involve  the  response  data,  y,  and  the  regressors  Xj,  Xj, 
etc.  All  too  often,  the  data  and  (a)  and  (b)  are  combined  into 

y  -  X0  +  E  (2) 

where  X  is  an  nxp  matrix  of  regressors,  possibly  including  the  constant 
carrier,  y  is  nxl,  Q  is  pxl  and  e  is  nxl.  The  estimated  coefficients,  b, 
are  then  obtained  by  a  fitting  process  (usually  least-squares )  without  a  great 


deal  of  thought. 


An  important  first  step  is  to  look  at  the  variables  y,  and  X^,  Xj,  etc. 
separately.  These  data  should  be  explored  using  histograms,  stem-and-lea f 
plots,  boxplots,  etc.,  (Velleman  and  Hoaglin,  1981)  and  granularity  (clumps, 
holes),  outliers,  and  asymmetry  ought  to  be  noticed.  Outliers  need  to  be 
tagged,  and  possible  transformations  considered.  Of  course,  some  outliers  mey 
not  be  so  prominent  when  we  consider  the  multivariate  nature  of  the  data. 
Transformations  considered  now  may  also  be  unnecessary  later,  but  asymmetries, 
outliers,  and  large  changes  of  magnitude  are  clues  that  some  variables  may  be 
in  the  wrong  units.  In  short,  take  a  hard  look  at  the  raw  lata.  Do  nothing 
if  you  wish,  but  set  up  a  list  of  things  to  check  as  you  go  further. 

The  response  variable,  y,  is  often  supposed  to  be  a  random  variable  with 
some  probability  distribution.  If  it  has  only  two  values,  looks  Poisson, 
etc.,  do  not  try  ordinary  least-squares  regression.  You  will  get  stupid 
results.  Consider  other  models  such  as  the  generalised  linear  models 
discussed  by  McCullagh  and  Nelder  (1983).  The  probability  plotting  techniques 
discussed  by  Chambers  et  al  (1983)  are  useful  for  checking  these  assumptions. 

When  a  thorough  univariate  analysis  has  been  done,  it  is  time  to  consider 
the  bivariate  (and  eventually  multivariate)  nature  of  the  data.  Plots  of  y 
versus  X^  are  always  worth  making,  especially  for  considering 
transformations  to  straighten  the  plot  and  computing  rough  correlations,  but 
can  be  misleading  if  used  to  develop  precise  models  because  of  the  effects  of 
other  regressors.  Bivariate  plots  of  the  regressors  are  also  useful  for 
finding  holes,  outliers,  etc.,  but  the  number  of  plots  increases  rapidly  with 
p.  However,  these  provide  the  first  clues  to  the  fact  that  information  in  the 
"design"  or  factor  space  may  be  spotty,  clumpy,  have  holes,  or  be  sparse. 
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Unfortunately,  we  cannot  do  this  well  in  higher  dimensions— at  least  not  yet. 

A  number  of  people  are  working  on  this  (Chambers  et  al,  1983,  Chapter  3). 

Usually  a  tentative  model  or  equation  comes  with  the  data.  While  doing 
our  exploring  we  should  see  if  the  model  is  sensible.  Possible  modifications 
should  be  noted  for  later  consideration.  It  is  always  tempting  to  make  g(x) 
linear  in  some  proposed  coefficients  (a  so-called  linear  model).  However, 
g(x)  may  be  quite  different— clues  to  this  effect  should  be  noted  for  they  may 
require  different  approaches.  The  model  could  be  nonlinear  in  the  parameters 
or  perhaps  a  non-parametric  approach  is  needed  (Friedman  and  Stuetzle,  1981). 
The  diagnostics  to  be  discussed,  below  assume  that,  at  least  tentatively,  the 
model  in  (2)  is  considered  reasonable.  Diagnostics  may  open  our  eyes  to 
further  problems, but  they  cannot  take  the  place  of  a  good  preliminary  look  at 
the  data,  structural  model  (g(x)  or,  more  specifically,  XQ)  and  stochastic* 
model  (distribution  of  y  or,  in  some  cases,  e)«  When  diagnostics  point  to 
changes  we  should  use  these  preliminary  procedures  after  making  changes  and 
again  apply  diagnostics  to  see  if  we  have  improved  our  analysis. 

PARTIAL  REGRESSIONS  AND  PLOTS 

Most  of  us  are  used  to  thinking  about  the  least-squares  estimates  as 

b  -  (XTX)“lXTy.  (3) 

However,  it  is  often  more  instructive^to  think  about  the  estimated 
coefficients  in  a  different  way.  Denote  the  residuals  found  when  y  is  fit  by 
all  but  the  jtfl  regressor  by 
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Thus  y,[j]  ifl  the  vector  of  least-squares  residuals  obtained  by  regressing  y 

on  all  regressors  except  the  j*1*1  and  is  often  called  the  adjusted  response 

variable.  Similarly,  let  X.  m  denote  the  residuals  obtained  by  regressing 
-  j  •  l  J  J 

Xj  on  all  of  the  remaining  regreasora.  These  are  called  the  adjusted 
regressors.  It  is  not  hard  to  show  (Mosteller  and  Tukey,  1977,  p.  344)  that 

n 

iM 

b.  «  -  (5) 

J  ?  2 

k.i  Xkj.[jl 

where  Xjj  jjj  is  the  ith  element  of  the  vector  Xjt[j]  *n<*  Xi,[j] 

f*  h 

the  i  element  of  the  vector  y  j j  j .  This  formula  should  be  compared  to 
that  for  simple  linear  regreasion  through  the  origin.  A  great  deal  of 
information  about  bj  can  be  obtained  by  plotting  y  jjj  against 
for  each  j.  These  are  called  partial  regression  plots  (or,  in  some  cases, 
adjusted  variable  plots).  Useful  references  are  Belsley,  Kuh,  Welsch  (1980) 
and  Chambers  et  al  (1983).  Both  of  these  contain  interesting  examples. 

Some  properties  of  these  plots  are: 

(a)  The  least-squares  linear  fit  to  the  plotted  data  has  slope  "  bj  and 
intercept  ■  0  (when  j  is  not  the  intercept  variable). 

(b)  The  residuals  from  the  least-squares  linear  fit  are  the  final 
multiple  regression  residuals,  y-Xb. 

(c)  It  is  relatively  easy  to  see  how  individual  data  values  influence  the 
estimation  of  bj. 

(d)  Often  some  information  about  nonlinearity,  heteroscedasticity  and 
unusual  patterns  can  be  obtained. 
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An  example  of  a  partial  regression  plot  is  given  in  Figure  1.  The  simple 
linear  regression  line  is  included  and  some  Interesting  points  have  been 
marked . 


Until  recently,  partial  regression  plots  were  thought  to  be  hard  to 
obtain.  Velleoan  and  Welsch  (1981)  show  that  this  is  not  the  case. 


cT  ■  cx^rV1 

b  »  CTy 


bj  “  ^Uti¬ 


lising  the  normal  aquations  and  (S),  we  can  show  that 


n 

Z  x 


j  n 


Skj.[J] 


The  uniqueness  of  the  least-squares  estimates  implies  that 


'ij  n 


lc-1 


or  equivalently 


Xij.[j]  n  2 

k-l 

Furthermore,  Hosteller  and  Tukey  (1977)  have  shown  that 


y. t ji  ’  • 4  Vj.tj) 


where 


e  “  y  -  Xb. 
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Therefore  b,  C,  and  e  are  all  we  need  to  get  partial  regression  plots.  A  well- 
organised  regression  program  can  obtain  C  very  easily.  There  is  no  excuse  for 
not  making  these  plots  a  part  of  every  regression  analysis.  They  are  an 
essential  diagnostic  tool. 


C0LLHKA1XTT 


Before  going  further,  it  is  advisable  to  get  some  feeling  for  collinearity . 
A  quick  way  is  to  note  from  (9)  that 

_2 


var  (b.)  -  — - 

J  ® 


(14) 


If  the  sum  in  the  denominator  is  small  compared  to,  say,  £  x. ,,  then  x.  has  been 

i-1  3 

well  fit  by  the  other  regressors.  Since  we  are  used  to  centering  our  data,  we 
often  compute  the  squared  multiple  correlation  of  Xj  on  the  other  regressors, 


2  £-1  Xij-£J1 

r:  -  i  -  - 

J  n  -  9 

Z  (*,  ,”x  ,)  * 

£■1  J 


(15) 


,th 


where  x.  is  the  j  variable  mean.  Note  that  if  we  are  interested  in  using 

J 


models  with  an  explicit  intercept  rather  than  centered  data,  the  denominator 

.2 


of  Rj  should  not  be  centered.  One  statistic  often  proposed  as  a  measure 
of  collinearity  is  the  variance  inflation  factor.  VIF,  found  from 


V1F.  - 


If  we  just  had  a  simple  linear  regreasion  on  Xj, 


y  ■  c  +  dX j  +  e  , 


Chen 


var  (d)  " 


-  J 


Z  (x..  -  x.) 

J 


Thus 

var  (b.)  -  var  (d)  •  VIF.,  (17) 

J  j 

and  we  see  Chat  VIF  measures  Che  variance  inflation  due  to  the  presence  of 
additional  regressors. 


The  drawbacks  of  the  above  approach  (VIF.  large)  are  that  it  does  not 

J 

tell  us  which  regressors  are  involved  or  how  they  are  involved  in  the 
collinear  relation  with  X j .  A  very  useful  way  to  get  this  information  is 
described  in  Chapter  3  of  Belsley,  Kuh,  and  Walsch  (1980). 

If  collinearity  appears  to  be  a  problem,  it  is  wise  to  reduce  it  as  much 
as  possible  before  doing  influential  data  diagnostics.  When  the  diagnostics 
that  follow  suggest  altering  or  setting  aside  data  it  is  essential  that  the 
new  model  be  rechecked  for  collinearity. 


One  of  the  preliminary  steps  for  regression  analysis  that  we  have 
discussed  is  the  making  of  bivariate  plots  of  the  regressors.  Generally,  the 
eye  will  notice  outlying  points  in  these  plots.  However,  it  is  hard  to  make 
higher  dimensional  plots  and  a  variety  of  tools  exist  to  overcome  this 
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problem.  One  of  the  easiest  to  use  measures  a  type  of  distance  from  x>  the  row 
vector  of  regressor  means ,  to  each  observation  : 

<x£  -  x)<XTxf1(xi  -  x)T.  (18) 

Here  X  is  the  X  matrix  without  the  intercept  column  and  with  column  means 
subtracted  off  of  the  remaining  columns.  Similarly,  (Xj  -  x)  omits  the 

intercept  element.  Belsley,  Kuh,  and  Welsch  (1980)  show  that 

1  _  eel  -  T 

h, - ■  (x,  -  x)(X  X)  (x.  -  x) 

ini  1 

where 

h£  -  x£(XTX)“lxJ 

is  the  diagonal  element  of  the  projection  or  hat  matrix, 

T  -IT 

H  -  X(X1X)  V, 

so-called  because 

y  *  Hy  . 

Most  modern  regression  programs  now  compute  ht  and  I  would  not  use  one  that 
failed  to  do  so. 

Often  we  will  want  to  hypothesise  that  x^  is  possibly  erroneous  or 
"strange."  Then  we  may  wish  to  measure  the  distance  from  to  the  rest,  of 
the  data.  One  useful  way  to  do  this  is  to  compute. 

dt  ■  (xt  -  ^(i))(X<i)TX(i))“1(xi  -  x  (i))T 
where  x  (i)  and  X  (i)  are  obtained  by  assuming  that  the  i  observation  did 


(19) 

(20) 

(21) 

(22) 


not  exist. 


Since  is  related  to  Mahalanobis  distance,  it  is  not  hard  to  show 
(Belsley,  Kuh,  and  Welsch,  1980)  that 

k-1! 

“i  [  1  -  hjj  n-1  * 

Both  h^  and  h^/d-h^)  will  prove  to  be  ueeful  in  what  follows. 


Hoaglin  and  Welsch  (1978)  discuss  many  propertiea  of  h^.  In  particular, 

n 

0  <  h.  <  1  (or  1/n  <  h.  <  1  when  an  intercept  is  present)  and  I  h.  *  p  io 
-  i-  “  1  “  i-1  1 

that  the  average  value  is  p/n.  Let  b(i)  denote  the  least-sqaures  estimates 

t  h 

obtained  without  using  the  i  observation.  Then  simple  algebra,  e.g.  (32), 
shows  that 

y^  ■  x^b  ■  (1  -  h£>Xjb(i)  +  h^y^.  (2 

A 

Thus,  y^  is  a  convex  combination  of  the  prediction  x^b(i)  and  the 

observation  y^.  The  ratio  h^/d-h^)  determines  the  relative  contribution  of 

fch  * 

each  part.  When  h^  is  one,  the  i  observation  completely  determines  y^  and 

h  m  V 

There  is  no  general  agreement  on  when  h£  is  "large."  Hoaglin  and  Welach 
(1978)  argue  that  an  individual  h^  should  not  be  too  far  from  a  balanced 

a*  W 

design  (all  h^  ■  p/n)  and  call  the  i  observation  a  leverage  point  when 

h^  >  2p/n  (provided  n  >  2p).  Belsley,  Kuh,  Welsch  (1980)  show  that  when 

the  (rows  of  X)  are  i.i.d.  multivariate  Gaussian,  the  distribution  of 

h^/d-h^)  can  be  related  to  an  F-statistic.  This  leads  to  a  criterion 

th 

that  calls  attention  to  the  i  observation  if  h^  >  3p/n.  Note  that 
these  leverage  criteria  depend  on  p  and  n. 


Huber  (1981)  uses  (25)  and  suggests  that  when  h;  >  0.5,  special 
attention  is  called  for  and  observations  with  h^  >  0.2  should  be  noted. 

These  leverage  criteria  are  independent  of  p  and  n. 

A  useful  compromise  between  these  two  general  approaches  is  to  consider 
hj ,  i"l,...,n  as  a  batch  of  data  to  be  analysed  by  exploratory  data  analysis 
(Velleman  and  Hoaglin,  1981).  Observations  with  outlying  values  of  h^  would 
then  be  considered  leverage  points.  My  own  simple  rule  of  thumb  is  to  pay 
attention  when  h^  >  min  (0.2,  3p/n). 

Note  that  if  all  of  the  data  is  replicated  m  times,  then  the  new  value  of 

* 

h-  is  the  old  value  divided  by  m.  Cut-offs  that  depend  on  the  sample  size 
(such  as  3p/n)  adjust  for  this  so  that  replication  does  not  affect  those 
points  we  determine  to  be  outlying.  This  seems  to  us  to  be  a  useful  property, 
especially  when  hj  is  related  to  distance  measures. 


It  is  also  possible  to  compute  the  contribution  of  the  individual 
regressors  to  the  leverage  of  each  observation.  Let 

h  -  htj3 

where  h^j  is  the  vector  of  leverage  values  when  X^  is  omitted  from  the 
regression  model.  The  partial  leverage,  rij  can  be  found  from 

2 


(26) 


'ij 


liLlil 


(27) 


l  x 
k-1 


j) 


t  h 

which  is  the  leverage  of  the  i  point  in  the  partial  regression  plot  for 
bj.  Data  points  with  large  partial  leverage  for  a  regressor  can  exert  an 
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I 


undue  influence  on  the  selection  01  that  regressor  in  most  automatic  regression 
model  building  methods.  For  some  examples,  see  Henderson  and  Velleman  (1981). 


RESIDUALS 


While  looking  for  leverage  points  is  a  relatively  new  tool,  examining 
various  plot's  of  the  residuals,  e^,  is  not.  Surely  residuals  should  be 
plotted  against  index  (or  time),  against  fitted  values,  against  proposed  new 
regressors  (it  is  best  to  adjust  the  new  carrier  for  those  already  in  the 
model  by  using  the  residuals  from  xney  regressed  on  the  current  model), 
etc.  Probability  plots  should  also  be  made.  An  excellent  discussion  is 
contained  in  Chambers  et  al  (1983). 


We  feel  that  the  residuals  should  be  properly  scaled.  Since 
var  (e£)  ■  o(l-h.),  two  useful  choices  are  the  internally  studentised 
residual 


and  the  externallv  studentised  residual 


1  s(i)/TIh7 

1  n  2 

where  s  is  the  standard  error  of  the  regression  (~ —  £  (y,-x,b)  )  and 

n"p  i-1  1  1 

s(i)  is  the  same  but  with  the  i6*1  observation  omitted.  A  simple  formula 
relates  s  and  s(i): 


(n-p)s2  ■  (n-p-l)s2(i)  +  * 


Under  the  usual  Gaussian  error  assumptions,  has  a  t-distribution  with 
n-p-1  degrees  of  freedom.  If  a  dummy  variable  with  zero  in  all  positions 
except  for  a  one  in  the  I**1  position  is  added  to  the  current  model  (X),  then 

it 

is  a  useful  diagnostic  for  seeing  if  there  should  be  a  shift  in  the 
intercept  for  the  ifch  observation.  Further  details  are  contained  in 
Belsley,  Kuh,  and  Welsch  (1980). 


Another  form  of  residual  is  often  useful.  The  predicted  residual  is  found 
by  computing 


yi‘  xib(1)  ‘  omT)  • 


(31) 


Hoaglin  and  Welsch  (1978)  have  noted  that  when  (31)  is  scaled  by  its  standard 

it 

error,  the  result  is  just  e^. 


MEASUKK  Of  XVTLUnCE 


Looking  for  leverage  points  and  examining  various  types  of  residuals  form 
an  important  step  in  regression  analysis.  However,  we  would  like  to  know  if 
an  observation  is  having  a  disproportionately  large  impact  on  our  analysis. 

An  observation  is  called  influential  if  its  deletion  woi  .d  cause  major  changes 
in  estimates,  confidence  regions,  test  and  diagnostic  statistics,  etc.  Usually 
influential  observations  are  outside  the  patterns  set  by  the  majority  of  the 
data  in  the  context  of  a  regression  model  (including  the  structural  model, 
stochastic  model,  and  fitting  procedure).  Influential  data  usually  arise  from 
errors  in  observing  or  recording  data,  structural  model  failure  (for  example, 
nonlinear  instead  of  linear)  and  legitimate  extreme  observations.  Deletion  is 
a  way  to  find  procedures  to  measure  influential  data.  Data  should  not  be 


deleted  because  they  are  influential,  but  should  be  flagged  and  carefully 
examined.  Alternative  fits  or  forecasts  may  be  needed,  one  with  and  one 
without  these  data.  Judgment  or  information  external  to  the  data  will  often 
be  necessary. 


There  are  many  ways  to  measure  influence.  Perhaps  the  most  common  is  to 
think  of  all  of  the  data  but  the  1  observation  as  "good"  and  the  i  as 
potentially  "strange."  We  want  to  find  an  influence  function  or  measure  to 
see  if  the  i*^  observation  really  is  a  cause  for  concern.  A  very  useful 
influence  function  is 


b-b(i) 


,vT„rl  T 
(X  X)  X£S£ 

(X^rVJCy 


x^b(i)) 


(3?) 

(33) 


or  for  each  estimated  coefficient 


x. .  . . ,  (y.-x.b(O) 

b.-b.d)  ■  UfUl-l-i - 

J  J  ?  2 

“kj-Ul 


(34) 


This  can  also  be  stated  in  terms  of  (11)  rather  than  adjusted  variables. 


It  is  often  convenient  to  scale  this  measure  in  some  way.  Since  we  are 
usually  interested  in  changes  in  the  estimated  coefficients  that  are  a 
substantial  fraction  of  the  stochastic  variability  of  b,  we  divide  by  the 

standard  error  .of  b..  To  estimate  the  standard  error  we  use 

J 

A 

o  “  s(i),  since  we  would  like  an  estimate  of  o  that  is  not  subject  to  the 


X) 


jj 


with 
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There  ere  three  basic  weys  to  decide  when  iDBETASl  ere  large.  The  fir  t 

is  to  note  which  ones  ere  lerger  then,  eey,  0.5  or  1.  Thet  ie,  setting  eside 

one  obaervetion  ceueee  e  0.5  or  1  etenderd  error  chenge  in  the  eatimetion  of 

fl..  A  second  method  (Beisley,  Kuh,  end  Welsch,  1980)  usee  the  feet  thet 
J 

when  c^j  is  cons  tent  for  ell  i  end  the  h^  ere  belenced, 

Z  (DBETAS. .)2  «  1. 
i-1 

When  IDBETA8 ^ jl  is  greater  then,  sey,  twice  the  everege  value  l//n,  we  teke 
note.  A  precticel  rule  of  thumb  is  to  use  m.in  (0.5,2//n)« 


The  third  approach  is  to  look  et  the  DBETAS  vie  exploratory  date  enelysis 
or  contour  plota.  For  e  fixed  j,  DBETAS ^  consists  of  the  product  indicated 
in  (36).  We  plot  the  pertiel  leverage  portion  on  the  x-axis  end  the  predicted 
residual  part  on  the  y-axis.  Contours  of  constant  Influence  x.y  ■  c  ere  also 
plotted.  Figure  2  show*  inch  a  plot  for  c  "0.5,  1.0,  1.5,  etc.  The  symbol  + 


denotes  a  positive  DBETAS  and  A  a  negative  DBETAS .  Some  potentially 
influential  points  have  been  tagged. 


Often  we  are  more  interested  in  predictions  then  coefficients.  A 

prediction  is  just  a  linear  combination  of  the  estimated  parameters,  say. 

T  T  T 

A  b .  It  is  natural  to  compare  A  b  to  A  b(i)  and  scale  with  a 

measure  of  the  standard  errors  of  the  fit.  sCiV^.  However,  we  often  do 
not  know  A  so  we  look  for  the  worst  case 

ffiT|»  VV1^-  - 

i  .  <i»  <rx)  t  .  (i) 

hl  ,  *.2 

From  (25)  we  also  note  that  the  difference  between  the  fit,  Xjb,  and  the 
predicted  fit,  x^U),  is  just  h^/U-h^).  When  scaled  by  a  measure 

of  standard  error  of  the  fit,  sCO/hi,  we  get 


(38) 

(39) 


x.b-x.b(i) 
DFITS,  •  — - - 

1  sU)/K7 


A  \l/2  ,  * 
<1=5}  *i 


(40) 


which  is  the  square  root  of  (39).  Notice  that  DFIT8  is  the  product  of  a 
leverage  factor  (24)  and  the  externally  studentised  residual  (28). 

Again,  there  arc  a  number  of  approaches  to  deciding  when  | DFZTS I  is 
large.  We  can  use  a  fraction  of  standard  error,  like  0.5,  1,  etc.,  or  note 
that 


n 

Z  DFITS 
i-1 


2  z 


(41) 


when  h{  ~  p/n.  We  do  not  want  any  observation  to  stray  too  far  from  the 
average  influence  so  we  would  single  out  observations  with  I DFITS  £  I  >  2/pA/n. 
A  reasonable  rule  is  to  use  min  (1,  2/p//n)  as  a  cutoff.  Cook  and  Weisberg 
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(1982)  develop  e  statistic  similar  to  DFITS^,  namely 

.  2 

d2  -  Ir-i-)  •  -!i _  a 

Di  pU-h^  ,2(i-h  )  *  ^ 

o 

They  suggest  that  D,  nay  be  eonaidered  large  when  it  exceeds 

F^'^  or,  approximately,  one.  This  aeems  to  be  an  unduly 
p,n-p 

conservative  cutoff  in  practice,  D?'  is  also  troublesome  because  it  uaee 

s2  instead  of  s2(i)  and  hence  ia  not  robust  to  errors  in  the  ith 

2 

obaervation.  Sea  Wslsch  (1982)  for  further  diecuesion  of  D  j. 


1/2 

We  prefer  to  look  at  contour  plots  with  [h^/(l-h^)J  on  the  x-axis 

it 

and  ja^l  on  the  y-axis.  Constant  influence  contours  nay  be  plotted  as 
before.  Figure  3  provides  an  example. 


Cut-offs  baaed  on  sample  sise  adjust  for  the  fact  that  as  sample  siee 
increases,  variance  decreases  so  that  biaa  caused  by  erroneous  data  becomes 
the  major  factor  in  determining  the  mean  square  error.  To  control  this  bias 
relative  to  the  decreased  variance  requires  cut-offs  that  are  reduced  as 
sample  sice  is  increased. 

Often  in  inference  we  are  interested  in  confidence  intervals  or  regions. 
A  confidence  region  consists  of  a  center,  a  shape,  and  a  scale.  For  example, 
in  regression  a  confidence  region  might  be  all  8  satisfying 

(b-8)TXTX(b-fl)  <  # 

s* 

T 

where  a  is  based  on  the  F  statistic  and  p.  Here  b  is  the  center,  X  X  is  the 
2 

shape,  and  a  is  the  scale. 
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So  far  we  have  looked  at  diagnostics  for  the  center,  b.  To  see  what 
T  t  h 

happens  to  X  X  whon  the  i  observation  is  considered  suspect,  ve  can 
look  at 

trace  [x’xiJXtDXX1!)"1]  -  p-h{ 
or  a  ratio  of  volumes 

dat[XT(i)X(i)l  .  ,_h  ^ 
dot (XTX) 

These  equations  just  provids  more  reasons  to  look  at  h^  in  its  own  right 


As  for  scale,  we  note  again  that 

2 

(n-p)s^  ■  (n-p-l)a^(i)  + 


so  that 


s“  ~ 

- -  M  1  4.  - - - 

2  1  *  n-p 


(2 
s  ( i) 


Again  we  have  already  looked  at  a^  extensively. 


These  measures  can  be  combined  by  looking  at  the  ratio  of  covariance 
matrix  determinants. 


COVRATIO  - 

det  sZ(XTX)“ 1 


(45) 
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An  important  point  to  note  ia  that  observations  with  larga  h^  dacraaaa 
tha  a  ice  of  a  confidence  region  while  observation*  with  largo  |a^| 
increase  it.  Our  goal  should  be  to  insure  that  wa  are  alerted  to  potentially 
influential  observations.  As  we  can  see  from  the  above,  influential  observa¬ 
tions  can  be  both  useful  and  harmful.  Mow  we  treat  them  will  depend  on  the 
purposes  of  our  analysis  and  their  relation  to  the  rest  of  the  data  and  our 
models.  The  beat  rule  of  thumb  is  that  thare  may  bs  more  than  one  good  and 
valid  analysis  of  a  data  set.  Sometimes  an  analysis  with  an  influential 
observation  and  one  without  are  the  only  way  to  adequately  summarise  the  data. 


GKXE1ALIZKD  LllflAI  MODEM 

When  the  response  variable  y  is  Bernoulli,  binomial,  Poisson,  etc., 
generalised  linear  models  (GLM)  are  appropriate.  A  detailed  discussion  is 
contained  in  McCullagh  and  Nelder  (19S3).  Many  of  the  ideas  discussed  above 
can  be  carried  over  to  these  models  as  well.  Basic  references  are  Pregibon 
(1979,  1981). 
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The  essential  idea  is  to  find  an  influence  function  (b-b(i)),  for  the 
parameters  of  the  model.  In  the  GLM  case,  this  cannot  be  done  exactly  since 
the  computation  of  b  requires  the  solution  of  a  system  of  nonlinear  equations 
via  iterative  procedures.  However,  b-b(i)  can  be  approximated,  usually  by 
taking  one  iteration  away  from  b  (the  fully  iterated  solution)  with  the  ith 
observation  removed  in  an  appropriate  way.  Various  kinds  of  residuals  can  be 
defined  as  well  as  useful  plots. 

This  is  an  extremely  active  area  of  research  at  the  present  time, 
especially  generalizations  to  survival  analysis  (Hall  et  al,  1982), 
proportional  hazards  and  censured  data  (Cain  and  Lange,  1984),  Cox  models 
(Storer  and  Crowley,  1985),  matched  case  control  studies  (Fregibon,  1984,  and 
Moolgavkar,  et  al,  1984)  and  logistic  regression  (Johnson,  1985).  Tho 
bibliographies  in  these  papers  provide  a  good  overview  of  work  in  this  area. 

iKFLtmrrxAL  subsits  ov  data 

If  there  are  two  or  more  outliers  in  a  clump,  then  influence  functions 
based  on  setting  aside  one  of  the  observations  will  not  work  well  because  we 
will  see  little  change  until  the  entire  clump  is  set  aside.  The  methods 
discussed  above  generalize  to  subsets  of  data  (Belsley,  Kuh,  Welsch,  1980; 
Cook  and  Weisberg,  1982;  and  Welsch,  1982),  but  very  large  amounts  of 
computation  are  required. 

To  overcome  the  computational  problems,  we  have  developed  a  technique 
called  bounded-influence  regression  (Krasker  and  Welsch,  1982).  A  bounded- 
influence  estimator  can  be  viewed  as  a  procedure  to  find  data-dependent 


weights  (for  use  in  weighted  lesst-squares)  so  that  no  smell  subset  of  the 
data  is  overly  influential*  The  weights  and  related  statistics  then  become 
useful  diagnostic  tools.  Examples  and  associated  contour  plots  are  given  in 
Kraaker  and  Welsch  (1983).  Computational  details  are  discussed  in  Peters, 
Samarov,  and  Welsch  (1982). 

Another  promising  approach  uses  cluster  analysis  to  reduce  the 
computational  burden.  These  ideas  are  explored  more  fully  in  Cray  and  Ling 
(1984).  Kempthorne  (1984)  combines  clustering  with  direction  searches  to 
attack  these  problems. 

These  ideas  can  be  extended  to  generalised  linear  models  in  a  number  of 
ways.  Some  basic  references  era  Samuels  (1978),  Krasker  (1979),  Reid  (1981), 
Reid  and  Crepaau  (1985),  Pregibon(1982) ,  and  Accomando  and  Pagano  (1983). 

Much  mors  work  needs  to  be  done  in  this  area. 

COMPQXATIOI 

Computational  details  of  many  of  the  above  methods  are  treated  in  Velleman 
and  Welsch  (1981).  They  also  discuss  how  to  use  package  programs  such  as  SAS 
and  MINITAB  to  obtain  various  diagnostics.  The  plots  used  here  were  made  on 
the  TROLL  system,  a  large  data  analysis  and  modeling  system  available  under 
license  from  M, I.T, 

There  is  no  reason  why  good  diagnostics  should  be  omitted  from  a  packaged 
program.  They  are  essential  in  my  view.  We  can  all  demand  that  they  be  a 
part  of  the  new  generation  of  software  for  personal  computers  and 
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workstations.  Diagnostics  are  particularly  effective  on  these  devices  because 
graphical  tools  are  readily  available. 
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ABSTRACT.  Recent  changes  in  the  character  of  missile  systems  has  created 
the  situation  vhere  a  large  number  of  objects  may  be  located  in  a  relatively 
small  space  volume.  These  randomly  located  objects  are  identified  by  photo¬ 
graphing  the  space  volume  from  two  or  more  different  locations.  The  total 
number  of  objects  in  the  space  volume  is  unknown  and  the  number  of  objects  in 
the  intersection  of  any  pair  of  scenes  is  unknown.  The  objective  is  to  identify 
the  objects  that  are  in  the  intersection  of  two  or  more  scenes  .  It  is  also 
desired  to  correlate  the  objects  identified  in  two  or  more  sequential  observa¬ 
tions. 


I.  INTRODUCTION .  The  purpose  of  test  range  instrumentation  is  to  collect 
data  that  can  be  used  to  evaluate  the  performance  of  the  object  being  tested. 
Photography  has  been  a  standard  method  of  collecting  performance  data  since  the 
very  early  days  of  missile  development.  A  common  method  of  using  photography 
to  collect  performance  data  is  to  record  sequential  photographic  images  from  a 
telescope  mounted  on  a  precision  tracking  mount.  The  direction  of  a  line  con¬ 
necting  the  location  of  the  tracking  mount  and  the  target  is  determined  by 
correcting  the  direction  the  mount  is  pointing  (azimuth  and  elevation)  by  the 
amount  the  target  is  offset  from  the  center  of  the  field-of-view  of  the  tele¬ 
scope.  This  yields  azimuth  and  elevation  angles  for  the  target.  If  two  or 
more  telescope  equipped  tracking  mounts  observe  a  target,  then  the  position  of 
the  object  in  space  can  be  estimated  by  computing  the  point  that  minimizes  the 
sum-of-squares  of  the  distances  from  the  lines  defined  by  the  azimuth  and 
elevation  angles  from  each  mount. 

Recent  developments  in  self-contained  munitions  have  created  situations 
vhere  large  numbers  of  objects  are  expected  to  be  in  the  field  of  view  of  two 
or  more  telescopes.  It  is  possible  to  estimate  the  direction  of  the  lines 
connecting  the  tracking  mount  and  each  target  detected  in  the  field  of  view. 

A  large  number  of  objects  in  the  field-of-view  tends  to  complicate  the  process 
of  producing  data  that  can  be  used  to  evaluate  the  performance  of  the  objects 
being  tested. 

II.  Object  Correlation.  Techniques  for  performing  two  types  of  object 
correlation  are  desired.  First,  an  efficient  technique  for  identifying  the 
location  in  space  of  the  objects  in  the  field  of  view  for  a  single  scene.  The 
images  observed  in  a  photograph  may  not  represent  a  single  object  since  it  is 
possible  that  two  or  more  objects  may  share  the  same  direction  from  an  individ¬ 
ual  mount.  Due  to  differences  in  aspect  angle,  range,  or  pointing  direction, 
the  objects  photographed  from  one  mount  may  not  all  appear  in  the  field-of-view 
from  another  mount.  An  algorithm  for  this  application  must  consider  these 
factors  . 

\ f ter  the  objects  have  been  identified  at  each  individual  point  in  time, 
it  xs  necessary  to  correlate  individual  objects  as  a  function  of  time  in  order 
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to  estimate  velocity.  We  are  given  a  large  number  of  objects  traveling  in 
different  directions  and  at  different  velocities.  We  sample  the  location  of 
these  objects  at  discrete  intervals  in  time.  The  number  of  objects  identified 
at  each  time  interval  may  be  incomplete.  A  procedure  is  needed  that  correctly 
Identifies  each  Individual  object  in  each  scene  for  the  purpose  of  estimating 
its  flight  performance.  Bounds  on  the  expected  range  of  velocities  and  direc¬ 
tions  of  the  objects  of  interest  can  be  assumed. 

Ill.  Summary.  Methods  that  can  be  used  to  develop  estimates  of  the  flight 
performance  of  a  large  number  of  objects  from  time  sequential  observations  of 
space  position  as  sensed  by  photo  optical  techniques  is  required.  These  methods 
need  to  be  sufficiently  robust  to  function  when  some  of  the  data  is  incomplete. 


AN  INTRODUCTION  TO  FAILURE  MODE  STRATEGY  AND  EFFECTIVENESS  FACTORS 
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Abstract 


The  growth  potential  for  a  system  design  Is  defined  as  the  maximum 
reliability  that  can  be  attain  for  a  particular  management  strategy.  The 
management  strategy  toward  reliability  pTaces  failure  modes  into  two  groupsi 
those  that  are  fixed  when  seen  and  those  that  are  not  fixed  when  seen.  The 
management  strategy  also  determines  how  effective  the  fixes  are.  Consideration 
of  the  growth  potential  focuses  attention  on  the  Impact  of  not  fixing  certain 
failure  modes  and  quantifies  the  management  strategy  and  engineering  effort 
In  terms  of  attaining  the  reliability  objectives. 

In  this  paper,  we  discuss  a  management  strategy  framework  for  considering 
the  growth  potential  and  show  that  a  number  of  other  reliability  values  of 
Interest  during  a  development  program  can  also  be  addressed  within  the  same 
framework.  These  various  reliability  parameters  are  all  shown  to  be  related 
and  this  perspective  can  be  very  useful  In  the  planning  and  management  of  a 
reliability  program. 


Introduction 

In  the  development  of  a  complex  system,  the  Initial  prototypes  will 
generally  have  reliability  problems.  Consequently,  the  system  Is  typically 
subjected  to  a  development  testing  program  to  find  problems  and  Incorporate 
appropriate  corrective  actions.  This  process  may  Involve  making  reliability 
predictions  and  constructing  planned  growth  curves.  For  the  planned  growth 
curve,  an  Initial  reliability  value  must  be  determined.  In  addition,  such 
terms  as  inherent  reliability,  growth  potential,  the  requirement,  and  current 
reliability  values  all  play  a  role  In  the  design  and  reliability  growth  for 
the  system. 

In  this  paper,  we  give  a  practical  real  world  structure  for  putting 
Into  perspective  initial,  Inherent,  predicted,  growth  potential,  requirement, 
and  current  reliability  values.  All  of  these  values  are  shown  to  be  related 
In  terms  of  parameters  which  are  functions  of  the  design  strategy,  the  manage¬ 
ment  strategy  for  failure  modes,  and  effectiveness  factors.  When  used  with 
reliability  predictions  and  Failure  Modes,  Effects  and  Criticality  Analysis 
( FMECA) ,  these  results  are  useful  for  Initializing  and  constructing  planned 
growth  curves,  for  setting  realistic  requirements  and  for  developing  a  viable 
management  approach  to  reliability. 

Failure  Mode  Strategy 

The  reliability  values  of  Interest  In  this  paper  are  all  directly  related 
to  the  system  design  strategy  and  the  reliability  management  strategy,  In 
this  section,  we  discuss  the  management  strategy  In  terms  of  failure  mode 
classification  and  the  effectiveness  of  the  fixes. 


Among  other  things,  the  reliability  management  strategy  determines  what 
problems  seen  during  test  will  or  will  not  be  fixed.  The  management  strategy 
also  determines  how  effective  the  design  fixes  are  and  when  the  fixes  are 
Incorporated  Into  the  system.  Although  the  management  strategy  toward  reli¬ 
ability  growth  may  not  be  clearly  defined  or  formally  stated,  It  will  in 
fact  exist.  The  management  strategy  Is  determined  by  how  management  acts  In 
regard  to  reliability.  The  system  design  and  management  strategy  will  deter¬ 
mine  If  It  Is  possible  to  meet  the  reliability  requirement.  The  management 
strategy  should  be  considered  early  In  the  development  program.  In  addition, 
the  Impact  of  the  management  strategy  on  reliability  can  be  measured  from 
the  test  data  and  changed  If  necessary.  Numerical  examples  are  given  In  a 
later  section  for  evaluating  the  management  strategy. 

When  the  system  Is  tested  and  failure  modes  observed,  management  can  make 
one  of  two  possible  decisions,  either  not  fix  or  fix  the  failure  mode.  There¬ 
fore,  the  management  strategy  places  failure  modes  Into  two  categories  called 
Type  A  and  Type  B  modes,  Type  A  modes  are  all  failure  modes  such  that  when 
seen  during  test  no  corrective  action  will  be  taken.  This  accounts  for  all 
modes  for  which  management  determines  that  It  Is  not  economically  or  otherwise 
justified  to  take  corrective  action.  Type  B  modes  are  all  failure  modes 

such  that  when  seen  during  test  a  corrective  action  or  fix  will  be  attempted. 

♦ 

The  management  strategy,  therefore,  partitions  the  system  Into  an  A  part 
and  a  B  part.  Each  part  has  a  corresponding  failure  rate  and  mean  time  between 
failure  (MTBF).  See  Figure  1. 


Figure  1.  Management  Strategy  Partitions 
the  System  into  Two  Parts. 


Initial  Reliability 


At  the.  beginning  of  development  testing,  the  Initial  system  failure  rate 

ca  ^ 


where 


[2  -  that  failure  rate  due  to  type  A  failure  modes. 

(23  -  that  failure  rate  due  to  type  B  failure  modes. 

See  Figure  2.  The  actual  value  of  JTi  Is  determined  by  the  system  design. 

The  partition  Into  the  A  and  B  parts  is  determined  by  the  management  strategy. 


Figure  2.  Partition  of  Total  System 
Failure  Rate. 


The  system  MTBF  Is  1/ Failure  Rate.  The  Initial  MTBF  Is  generally  low 
relative  to  the  requirement  and  the  objective  Is  to  achieve  reliability 
growth  through  finding  problems  and  taking  subsequent  correction  actions. 


Current  Reliability  and  Effectiveness  Factors 


Reliability  growth  Is  achieved  by  decreasing  the  failure  rate  [5], 

The  failure  rate  (XJ  for  Type  A  failure  modes  will  not  change.  Wltn  the 
management  strategy,  reliability  growth  can  only  be  achieved  by  decreasing 
the  Type  B  failure  rate  [0.  It  Is  also  clear  that,  In  general,  we  can 
only  decrease  that  part  of  the  Type  B  mode  failure  rate  that  has  been  seen 
during  testing.  See  Figure  3. 


It  Is  very  Important  to  note  that  once  a  Type  B  failure  mode  Is  In  the 
system  It  Is  rarely  totally  eliminated  by  a  corrective  action.  After  a  Type 
B  mode  Is  found  and  fixed,  a  certain  percent  of  the  failure  rate  will  be 
removed,  but  a  certain  percent  of  the  failure  rate  will  generally  remain. 
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A  fix  effectiveness  factor  (EF)  Is  the  percent  decrease  In  a  problem 
mode  failure  rate  after  a  corrective  action  has  been  made.  A  recent  study 
by  the  US  Army  Materiel  Systems  Analysis  Activity  on  EFs  showed  that  an 
average  EF  d  was  about  70  percent.  That  Is,  on  the  average  about  30  percent, 
l.e.,  1-d  percent,  of  the  Type  B  mode  failure  rate  remained  in  the  system 
after  a  corrective  action. 


Figure  3.  System  Failure  Rate  Partition  at 
End  of  Test. 


Management  controls  the  resources  for  corrective  action.  Consequently, 
fix  effectiveness  factors  are  part  of  the  management  strategy.  For  the  Type 
B  mode  failure  rate  that  has  been  seen  during  development  testing,  we  will 
remove  d  percent  and  leave  1-d  percent  in  the  system,  For  Illustrative  pur¬ 
poses,  we  will  frequently  use  an  average  EF  d  to  be  70  percent  In  this  paper. 
Therefore,  after  the  corrective  actions  have  been  made,  the  current  system 
failure  rate  consists  of  the  Type  A  mode  failure  rate  plus  the  failure  rate 
for  the  unseen  Type  B  modes,  plus  30  percent  cf  the  failure  rate  for  the  * 

Type  B  nodes  which  have  been  seen.  See  Figure  4. 

An  Important  management  question  Is:  Car  the  requirement  ever  be  attained 
with  the  way  we  are  doing  business  l.e.,  with  the  management  strategy?  If  the 
requirement  can  be  met,  then  usually  we  also  want  know  how  long  it  will  take. 
This  car,  be  answered  by  addressing  the  growth  rate. 
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Figure  4.  System  Failure  Rate  After- 
Corrective  Actions. 


Growth  Potential  and  Requirement 

The  growth  potential  Is  the  maximum  reliability  that  can  be  attained  with 
tin t  system  design  and  reliability  growth  management  strategy.  Die  growth 
potential  will  have  been  attained  when  all  Type  B  failure  modes  have  been 
found  and  a  fix  Incorporated  into  the  system,  for  the  system  design  and 
management  strategy  this  Is  the  limiting  reliability.  The  growth  potential 
reliability  may  never  actually  be  achieved  In  practice. 

Figure  5  portrays  the  growth  potential  for  a  system  with  an  average  EF 
of  70  percent.  The  growth  potential  failure  rate,  assuming  an  average  EF  of 
70  percent,  consists  of  the  Type  A  failure  rate  plus  30  percent  of  the  Type 
B  failure  rate. 

The  Initial  failure  rata  QT  1'0r  the  system  at  the  beginning  of  develop¬ 
ment  testing  consists  of  the  failure  rate  [T]  for  the  Type  A  modesand  the 
failure  rate  QQ  for  the  Type  B  modes.  ThaFls,  [SJinlt  "  GO  +  ©• 

The  growth  potential  failure  rate  Is  the  most  that  the  Initial  failure  rate  can 
be  reduced  with  the  management  strategy.  It  is  the  best  failure  rate  attainable. 
For  an  average  EF  of  .70,  the  growth  potential  failure  rate  Is  expressed  by 

IUgP  ■  G3  +  ( *30)  [U 
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Figure  5.  Failure  Rate  Growth  Potential. 


The  Initial  MTBF  and  the  growth  potential  MTBF  are  shown  In  Figure  6. 

An  Important  consideration  Is  whether  or  not  the  requirement  Is  below  the 
growth  potential  MTBF.  If  the  requirement  Is  not  below  the  growth  potential 
then  the  requirement  cannot  be  attained  with  the  current  system  design  and 
management  strategy.  The  methods  presented  In  Ref  [23  may  be  used  to  easily 
estimate  the  growth  potential  from  test  data.  ' 
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figure  6.  Initial,  Rtquiremtnt  and  Growth 
Potential  MTBF. 
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Suppose  a  system  Is  tested  for  400  hours  with  42  failures.  According 
to  the  management  strategy,  10  failures  are  due  to  failure  modes  that  will 
not  receive  a  corrective  action.  That  Is,  these  are  Type  A  failures.  Also, 
according  to  the  management  strategy,  32  failures  are  due  to  failure  modes 
that  will  receive  a  corrective  action.  These  are  Type  B  failures.  See 
Figure  7. 


Figure  7  Management  Strategy  for 
Type  A  and  6  Modes. 


We  next  estimate  the  Initial  failure  rate  and  the  failure  rate  and  the 
failure  rates  for  the  A  and  B  partition. 

go  ■  m  +  E* 

The  estimates  for  the  Type  A  and  Type  B  failure  rates  are  the  respective 
number  of  failures  for  that  type  divided  by  400,  the  number  of  test  hours. 
Consequently  with  the  above  management  strategy  the  Type  A  failure  rate  Is 

o  -  10/400 

and  the  Type  B  failure  rate  Is 
GO  -  32/400. 

The  estimate  of  the  Initial  system  failure  rate  Is 

GDlnlt  ■  32/400  ■  42/400  or  an  estimate  of  the  Initial  MTBF  of  9.5  hours. 

If  we  assume  an  average  EF  of  70  percent,  then  under  the  above  management 
strategy  the  growth  potential  failure  rate  Is  estimated  by 


[SJGP  -  10/400  +  (.30)  32/400. 

This  gives  an  estimate  of  the  growth  potential  MTBFof  20.4  hours. 

With  this  management  strategy  and  these  data,  the  Initial  MTBFof  the 
system  Is  estimated  to  be  9.5  hours  and  the  very  best  MTBF  that  may  be  attained 
Is  estimated  to  be  20.4  hours.  If  the  requirement  Is  less  than  20.4  hours 
then  there  Is  the  possibility  of  It  being  attained.  However,  If  the  require¬ 
ment  Is  greater  than  20.4,  say,  25  hours,  then  It  Is  very  unlikely  the  goal 
can  ever  be  reached  with  the  present  management  strategy,  regardless  of  how 
much  testing  Is  conducted.  See  Figure  8. 


GROWTH  POTENTIAL 


Figure  8.  Estimated  Initial  and  Growth 
Potential  MTBF. 


If  a  requ1remt.it  of  25  hours  MTBF  Is  to  be  attained  with  the  system 
design,  then  the  management  strategy  must  be  changed.  This  would  entail  fix 
ing  more  problems,  l.e..  Increase  the  Type  B  modes,  and/or  Increase  the  ef¬ 
fectiveness  of  the  fixes.  However,  It  Is  noted  that  an  average  EF  of  about 
70  percent  appears  to  be  typical  for  many  types  of  systems,  although  Individ 
ual  EFs  may  be  larger.  Therefore,  a  very  large  average  EF  may  not  be  war¬ 
ranted. 


Exampl  e 


Suppose  that  an  average  EF  of  70  percent  Is  assumed  but  we  desire  to 
change  the  management  strategy  so  as  to  Increase  the  growth  potential  MTBF 
above  25  hours.  Again,  suppose  that  the  system  was  tested  for  400  hours 
with  42  failures.  With  the  new  management  strategy,  39  failures  are  due  to 
modes  which  will  receive  a  corrective  action,  l.e.,  Type  B  modes,  and  3 
failures  are  due  to  modes  which  will  not  receive  a  corrective  action,  l.e., 
Type  A  modes.  See  Figure  9. 
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Figure  9.  Management  Strategy  for  Type 
A  and  6  Modes. 


The  growth  potential  failure  rate  for  this  management  strategy  Is  estimated 
to  be 


0GP  -  3/400  +  (.30)  39/400 

or  an  estimated  growth  potential  MTBF  of  27.2  hours.  This  Is  a  valid  estimate 
of  the  growth  potential  If  this  management  strategy  Is  maintained.  This 
Includes  classifying,  In  the  long  term,  at  least  93  percent  of  the  failures 
to  Type  B  modes  and  no  more  than  7  percent  of  the  failure  to  Type  A  modes. 

Also,  an  average  EF  of  at  least  70  percent  must  be  achieved. 

It  Is  Important  to  note  that  the  growth  potential  does  not  estimate  the 
current  reliability  -  It  estimates  the  maximum  reliability  that  will  be  achieved 
when  all  Type  B  failures  modes  have  been  found  and  fixed  by  a  corrective  action. 
It  Is  showned  In  Ref  [1]  that  the  current  failure  rate,  after  corrective  action 
of  Type  B  modes  seen  during  test,  equals  the  growth  potential  failure  rate  plus 
a  correction  term  for  unseen  Type  B  modes  still  In  the  system.  An  estimated 
of  the  current  MTBF  utilizing  EFs  Is  obtained  by  applying  the  projection  model 
and  methods  discussed  In  Ref  [1]  and  Ref  [2]. 

Example 

Assume  again  the  situation  In  the  first  example.  In  this  case,  32  failures 
will  receive  fixes  at  the  end  of  400  hours  of  testlnq.  The  various  failure 
times  for  the  Type  A  and  Type  B  modes  In  this  example  were  generate  by  com¬ 
puter  simulation  and  discussed  In  Ref  [13*  In  Ref  [1],  these  data  were  used 
to  obtain  an  estimate  of  14.7  hours  for  the  current  MTBF.  See  Figure  10. 


GROWTH  POTENTIAL 


Figure  10.  Estimated  Initial,  Current  and 
Growth  Potential  MTBF. 

Inherent  and  Predicted 


The  Type  A  failure  inodes  are  determined  by  the  management  strategy.  The 
Type  A  group  accounts  for  all  failure  modes  for  which  management  determines 
that  It  is  not  economically  or  otherwise  justified  to  take  corrective  action. 
Economically  justified  Is,  of  course,  relative  to  the  design  strategy.  For 
example,  what  is  economically  justified  for  one  car  type  may  not  be  for  an¬ 
other,  depending  on  the  respective  design  strategies  and  objectives. 

An  Inherent  failure  mode  will  exist  whenever  the  failure  rate  for  that 
mode  cannot  be  economically  reduced  further  by  corrective  action.  An  Inherent 
mode  failure  rate  will  remain  In  the  system.  Ideally,  the  group  of  Type  A 
failure  modes  should  consist  only  of  Inherent  failure  modes.  If  the  Type  A 
group  Includes  modes  which  are  not  Inherent  -  1,e.»  can  be  economically  cor¬ 
rected  -  then,  this  Is  Indicative  of  bad  reliability  management.  Also,  If  the 
failure  rate  for  the  Initial  group  of  Inherent  modes  at  the  beginning  of  reli¬ 
ability  testing  Is  large  relative  to  the  requirement,  then  this  Indicates  a 
bad  system  design  for  the  requirement.  When  all  system  failure  modes  are 
Inherent  -  cannot  be  economically  reduced  further  -  then  the  system  has  at¬ 
tained  Its  Inherent  reliability. 

A  valid  reliability  prediction  should  generally  be  expected  to  address 
the  Inherent  system  reliability.  This  is  very  Important  because  of  the  rela¬ 
tionship  between  Inherent  reliability  and  the  growth  potential  reliability. 

If  the  management  strategy  is  sound  such  that  all  observed  failure  modes 
which  are  not  fixed  are,  In  fact,  already  at  their  Inherent  failure  rates, 
and  those  failure  modes  which  are  fixed  have  EFs  that  reduce  their  failure 
rates  to  the  Inherent,  then  the  growth  potential  failure  rate  equals  the 
Inherent  failure  rate.  This  observation  can  be  very  useful  for  developing 
planned  reliability  growth  curves. 
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Conclusions 


In  this  paper,  we  have  discussed  the  Initial,  current,  growth  potential, 
requirement,  inherent  and  predicted  reliabilities  and  their  relationships  In 
terms  of  the  design  strategy  and  management  strategy  for  the  system.  In  par¬ 
ticular,  the  initial  failure  rate  is  partitioned  into  two  parts, 

ODlnit  “  (S  +  HI 

For  an  average  EF  d  the  growth  potential  and  initial  failure  rate  are  related 
by 

Sgp  ■  S3  +  U-d)  CO 

or 


UGP  “  COlniL  -  d[ O* 

In  addition,  the  current  system  failure  rate,  when  all  Type  B  modes  which  have 
been  seen  during  test  have  received  a  corrective  action,  Is  related  to  the  qrowth 
potential  by 

HI  cur  *  GDgp  +  correction  term  for  unseen  Type  B  modes. 

In  term  of  MTBF,  the  requirement  must  be  below  the  growth  potential  MTBF  if 
it  Is  to  be  attained.  Also,  the  predicted  reliability  should  generally  be  ex¬ 
pected  to  address  the  inherent  reliability.  Therefore  for  planning  purposes, 
we  may  equate  the  two,  l.e., 

Q]lnh  *  0Pred. 

In  general,  the  Inherent  MTBF  will  be  greater  than  the  growth  potential,  depend¬ 
ing  on  the  management  strategy.  Under  a  sound  management  strategy  they  will  be 
the  same,  l.e., 

OEJlnh  -  0GP. 

Therefore,  it  follows  that  given  a  realistic  prediction  and  assuming  a  sound 
management  strategy,  then 

IUgP  *  GOPred. 

These  relationships  are  very  useful  for  developing  planned  reliability  growth 
curves  and  for  evaluating  the  realism  of  attaining  the  requirement  with  the 
management  strategy.  Figure  11  summarizes  these  relationships. 
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ABSTRACT 


This  paper  describes  a  methodology  for  obtaining  the  A  and  B 
material  design  allowable  values  using  an  extreme  quantile 
modeling  process.  The  allowables  represent  a  value  determined 
from  a  specified  probability  of  survival  with  95  percent 
confidence  in  the  assertion.  The  survivalprobabilities  are  .99 
for  the  A  and  .90  for  the  B  allowable.  The  required 
representation  of  the  small  portion  of  the  data  in  the  left  hand 
tail  of  the  distribution  was  obtained  from  the  following  models: 
the  two  parameter  Weibull,  two  parameter  exponential  and  the 
Bootstrap  method. 

Development  of  the  exponential  model  is  presented  in  detail. 

The  primary  effort  involved  determining  an  unbiased  estimator  of 
the  LCB  for  the  exponential  sample  order  statistic  corresponding 
to  the  required  quantile  value.  This  was  obtained  in  terms  of 
the  LCB  of  the  same  ordered  value  from  a  uniform  distribution  and 
the  MLE  of  the  exponential  parameters. 

Applying  the  Weibull  censored  data  analysis  presented  in 
Lawless  [1]  provided  an  effective  method  for  weighting  the  lower 
ordered  values  where  at  least  seventy-five  percent  right 
censoring  was  considered.  The  Bootstrap  [2]  method  was  applied 
in  order  to  obtain  the  variance  at  the  one  and  ten  percentile 


values.  In  the  case  where  data  was  limited  or  contaminated  an 
extrapolation  process  involving  an  extreme  value  model  was 
introduced.  Excellent  correlation  was  obtained  from  among  all 
three  models. 


INTRODUCTION 


The  inability  to  obtain  exactly  the  same  structural  properties 
from  all  specimens  obtained  from  a  manufactured  material  results 
in  a  relatively  large  variability  in  strength  measurements  when  a 
large  number  of  specimens  are  considered,  in  the  case  of 
designing  an  aircraft  structure  it  is  required  to 
design  such  that  a  minimum  stress  value  exists  in  critical 
locations  and  these  values  do  not  exceed  the  minimum  guaranteed 
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material  properties  (strength).  Obtaining  minimum  strength  values 
will  reduce  possibility  of  some  production  components  containing 
weaker  material  than  that  from  the  laboratory  test  element.  This 
guaranteed  minimum  strength  value  is  defined  as  the  design 
allowable  by  aircraft  design  engineers. 

Usually  the  measured  value  is  considered  acceptable  in 
estimating  the  population  parameters  for  predicting  population 
percentiles.  In  the  case  of  the  design  engineer  it  is  advisable 
to  have  a  prediction  which  will  determine  the  accuracy  of  tha 
percentile  eatimate  at  a  high  degrea  of  atatiatical  eonfidance. 
This  ia  the  correct  interpretation  of  an  allowable  value.  For 
example,  certain  military  atandards,  e.g.,  MIL-HDBK-513)  require 
material  property  data  to  bo  presented  on  an  A  or  B  allowable 
basis.  A  and  B  allowables  are  defined  by  the  probability 
statement  providing  a  95  percent  confidence  of  the  assertion  that 
probability  of  surviving  ths  A  sllowabls  vslue  is  <.99  and  <,90 
for  tha  B  allowable. 

AMMRC  is  involved  in  the  development  of  a  atatiatical  chapter 
for  the  MIL-17  HDBK  on  compoeite  material  in  aircraft  structural 
design.  The  chapter  will  include  methods  for  determining  the 
•sign  allowable  values.  The  inability  to  identify  the  proper 
atatiatical  model  from  limited  or  multi-modal  data  motivated  the 
authors  to  find  new  robust  models. 


This  paper  presents  original  methods  for  obtaining  an  accurate 
measure  of  the  above  mentioned  design  allowables  involving  lower 
tail  modeling.  Breiman,  Stone  and  Gins  [4]  have  discussed  the 
difficulties  existing  in  model  identification  when  very  small 
tall  probabilities  are  required.  This  is  the  result  of  parameter 
estimates  that  usually  are  obtained  from  data  in  the  central 
portion  of  the  distribution  where  most  failures  occur  leaving  the 
tail  region  limited  in  representation.  This  Is  unfortunate, 
since  the  relatively  small  amount  of  data  in  the  tail  region  is 
of  prime  importance  to  the  allowable  computation.  The  lower  tail 
modeling  presented  in  the  paper  avoids  central  region  modeling  by 
modeling  lower  ordered  values  of  the  distribution.  The 
exponential  and  Bootstrap  methods  involve  truncating  the  tail. 

The  Weibull  modal  uses  a  censoring  process  for  the  high  ordered 
strength  values. 

When  small  samples  or  multi-modality  prevents  reasonable  model 
Identification  it  is  necessary  to  either  apply  conventional 
non-parametrlc  methods  or  devise  some  scheme  similar  to  those 
advocated  by  the  authors.  The  primary  difficulty  In  the  extreme 
quantile  modeling  technique  involved  determining  tolerance  bounds 
on  tha  quantile  values  in  the  allowable  computation. 

The  Exponential  Extreme  Quantile  LCB  Estimator 

Breiman,  Stone  and  Gins  introduced  the  exponential  extrema 
quantile  model  which  la  the  foundation  of  our  research.  Tha 
method  for  estimating  the  LCB  of  an  extreme  quantile  presented 
here,  however,  was  developed  by  the  author a.  h  Monte  Carlo 
investigation  of  the  original  (Breiman, et.al.)  LCB  estimator 
disclosed  a  bias  of  about  3%  when  sampling  from  an  exptnantial 
distribution.  Since  an  exponential  distribution  satisfies  the 
extreme  quantile  model  exactly,  it  Is  desirable  that  the  LCB 
estimator  be  unbiased  in  this  situation.  This  paper  presents  a 
new  LCB  estimator  which  la  unbiased  whan  the  underlying 
distribution  is  exponential. 


!*  *tV 


Consider  a  sample  of  n  independent#  identically  diatcibuted 
random  vari  iblea  from  the  two- parameter  exponential  distribution 


F(Xiiia) 


c 


-(t-X) / a 


0  <  X  <  t 


X  >  T 


Define  the  lower  tail  aa 

*X(1)  •  X(2)  *  *  * '  *  *<a)  ^  *  *  —  n^2  * 

where  is  the  jth  sample  order  statistic.  Assuming  that 

F(0fr,a)  is  approximately  aero,  the  joint  density  of  the  lower 
tail  values  may  be  approximated  as  m 

*(!)•• '•  ^(m)(Xlr"  *  -^yja"®  e  1 

..  .  (2) 

.  #-*(t-\)/e  n“® 

for  t>x(b)>** ,>x(i).  Based  on  the  lower  tail  values#  the  maximum 
likelihood  estimators  of  the  exponential  parameters  are 

m-1 

i  1-1  o) 

It  -  X(!0)  4  lin(n/a) 

where  a  has  been  weighted  with  m-1  rather  than  m  in  order  to 
yield  an  unbiased  estimator.  The  maximum  likelihood  estimator  of 
the  qth  quantile  X  #8<  q  <1,  is 


-  t  -  hln(l/q)  -  X(a)  -  A  ln(a/nq). 


This  is  obtained  by  solving  F(Xq  |T,a)  for  Xq  and  rsplacing 
parameters  with  their  MLE's.  The  lower  95%  confidence  bound  on 
Xq  with  q  ».l  and  q«,01  determines  the  B  end  A  design  allowables 
respectively. 

If  the  quantile  of  interest  corresponds  to  a  sample  order 
statistic,  then  the  true  lower  confidence  bound  on  the  quantile 
may  be  obtained  by  applying  the  inverse  probability  transform  to 
the  LCB  of  the  corresponding  order  statistic  from  a  uniform 
distribution;  it  follows  that  for  this  situation  the  true 
allowable  may  be  readily  obtained. 

In  what  follows,  we  have  restricted  attention  to  the  B  . 
allowable,  i.e.  the  lower  95%  tolerance  limit  on  the  10%  point. 
Consideration  of  the  A  allowable  (tolerance  on  the  1%  point)  may 
be  made  in  a  similar  manner.  Further,  we  have  considered 
primarily  samples  of  sise  10k  for  k  Integer,  since  in  this  case 
the  required  limit  corresponds  to  the  LCB  on,  the  kth  order 
statistic. 

Let  X1,..,X|)  be  lid  F(.;t,s)  as  in  equation  1.  The  density  of 
the  kth  order  statistic  is 

■  2^#-(»-k+l)(T-t)/a  <1-#.(T.t)/aj(k-l)  (5) 

and  the  true  B  allowable  may  be  expressed  ,es 

I  fY  (t;T,a)dt.  ■  .05. 

Jb  00  t6) 

with  change  of  variable 

I  u  . 

lv 

(7) 
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The  upper  limit  on  the  integral  is  tharsfors  ths  lowsr  5t  point 
of  tha  Beta(k,n-k+l)  distribution,  i.e.* 

'EC:lK  (1-vk'‘  ■  <»> 

■■o 

Tha  allowable  eatimator 


*b  "  X(m)  4  4  in(10ub»/n)  (10) 

may  be  obtained  from  equation  7  by  replacing  parameters  with 
their  MLE'8. 

lines  tha  MLE*s  sre  unbiased  in  this  ease,  we  have  derived  an 
estimator  which  will  provide  unbiased  allowable  estimate*  when 
the  underlying  distribution  is  exponential,  this  estimator  is 
computationally  very  simple)  the  percentiles  of  the  beta  edf 
(equation  9)  are  easily  obtained  on  a  computer.  We  have,  for 
ease  of  refarence,  celled  this  new  estimator  the  reduced  bias 
estimator,  or  EBB. 


A  Monte  Carlo  study  using  the  two  parameter  Weibull 
distribution  was  performed  in  order  to  provide  a  preliminary 
assessment  of  the  RBE.  Since  the  problem  of  determining 
allowables  is  particularly  acute  for  small  samples,  we 


considered  sample  sizes  of  10,  20, and  30,  with  tail  sizes  of  3,  6 
and  10  respectively.  The  Weibull  shape  parameters  (a)  considered 
are  In  the  range  2  <  a  <  i&0,  s  range  of  values  consistent  with 
those  expected  from  test  results  on  most,  materials.  This 
investigation  is  summarised  in  Figure  1, 

The  exponential  tail  allowable  estimator  is  s  linear 
combination  of  the  sample  order  statistics.  For  the  order 
statistics  of  a  Weibull  sample,  the  mean  and  covariance  matrix 
may  be  determined  exactly  [51*  Unfortunately,  the 
covariance  matrix  is  quits  complex,  therefore  a  simulation  was 
used  to  estimate  the  variance  of  the  allowable  estimator  from 
Weibull  samples. 

The  exact  expression  for  the  mean  of  the  kth  order  statistic  is 


*<W 


r(l*l/o)'(*n 


wEwfcssy 

■■0 
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Although  equation  11  appears  to  be  relatively  simple  to  evaluate, 
machine  roundoff  error  prevented  computation  of  the  sum.  the 
reason  for  the  difficulty  is  that  equation  11  is  proportional  to 
the  (k-l)st  backward  difference  of  (l/n)^41^  that  is 


Uk“1((l/n)(l4l/o)) 


I  WUsS) 

s«0 


(1+1/tt) 


(12) 


where  7  is  defined  by 

Vf(x)  ■  f(x-l)  -  f(x)  . 


The  difference  operator  may  be  expressed  in  terms  of  the 
derivatives  of  the  function  and  Stirling  numbers  of  the  second 
kind  (6). 

The  results  of  the  Honte  Carlo  study  are  presented  in  Figure  2, 
The  estimated  bias  is  expressed  as  a  percentage  of  the  allowable 
estimate.  As  a  measure  of  the  variance,  the  estimated  001 


51 


confidence  interval  it  alto  given  at  a  percentage  of  the 
estimate.  These  results  indicate  that  the  bias  is 
unacceptably  large  for  small  shape  parameter  but  decreases 
sharply  as  the  parameter  increases.  This  is  also  true  for  the 
confidence  interval  as  a  percentage  of  the  estimate.  The  mean 
square  error  decreases  with  increasing  sample  site. 

In  figure  3 ,  the  estimated  B  allowable  for  the  welbull 
distribution  is  plotted  vs  the  the  shape  parameter.  Note  the 
nearly  vertical  slope  of  the  curve  for  small  a,  The  estimated 
allowable  curve  agrees  closely  with  the  true  curve  over  the 
entire  range)  the  distance  between  the  curves  measured  along  a 
normal  is  always  small.  When  measuring  the  bias  and  confidence 
for  small  «»  we  are  cutting  the  curves  obliquely  in  a  region  of 
rapid  change  In  the  allowable  with  increasing  shape  parameter. 

In  order  to  compare  the  BBS  method,  a  standard  nonparametrie 
estimator  was  applied  •  the  first  ordered  vilue  from  a  sample  of 
30.  That  is,  if  the  unspecified  continuous  underlying 
distribution  is  f ( •) »  then 

p(x(1)<x.l)  -  P(P(X(1))<FCXiX))  •  P(P(X(l))<.l) 

■  1  -  .930  ■  .93  8 

Since  minimal  assumptions  are  made  with  respect  to  distribution, 
this  nonparametrie  estimator  generally  is  conservative.  In 
figure  4,  the  KBR  bias  and  confidence  for  a  sample  of  30  are 
repeated,  along  with  the  corresponding  exact  results  for  the 
first  ordered  sample  value,  except  for  very  small  shape 
parameter,  the  proposed  estimator  has  much  smaller  mean  square 
error  than  the  ether  nonparametrie  estimator. 


Determining  a  Tall  Truncation  Point 

A  problem  to  be  addressed  is  the  choice  of  a  tail  truncation 
point  (m  value),  in  the  simulation  just  described  the  tall  is 
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approximately  1/3  of  the  data.  This  choice  was  made  subjectively 
before  the  analysis  began.  To  examine  the  sensitivity  of  bias  to 
m  value,  the  expression  for  the  exact  expected  order  statistics 
(equation  11)  was  used  to  calculate  the  expected  bias  in  the  RBE 

for  the  sample  sizes  and  Weibull  populations  of  figure  1. 

* 

These  results  are  presented  in  Figure  5  a,  b,  and  c  for  sample 
sices  10,  20  and  30  respectively.  These  plots  show  considerable 
fluctuation  in  bias  for  small  shape  parameters.  The  bias 
decreases  steadily  with  increasing  modulus,  in  good  agreement 
with  the  Monte  Carlo  results.  From  Figure  5,  it  appears 
that  the  optimal  tail  lengths  are  3,  6,  and  8  for  sample  sices  of 
10,  20,  and  30  respectively.  For  small  shape  parameters,  a  longer 
tail  greatly  reduces  the  expected  bias.  Figure  5d  is  a 
magnification  of  Figure  Scs  it  illustrates  the  expected  bias  for 
a  sample  of  size  30  and  a  modulus  of  less  than  10.  If  one  takes 
a  tail  of  12  pointa,  the  expected  bias  may  be  reduced  to  51  or 
less  for  3<  &  <10.  A  more  complete  treatment  of  the  tall 
truncation  problem  will  be  presented  In  a  forthcoming  AMMRC 
report. 

Allowable  Estimate  From  the  Bootstrap  Method 


The  Bootstrap  method  used  in  determining  B  design  allowable 
values  involves  letting?  be  the  empirical  distribution  of 
observed  values  (strength  test  results)  ,i .a.  the  probability 
distribution  with  mass  1/n  for  each  X  value  (X^,  X2»  Xj...Xn). 
The  Bootstrap  sample  is  obtained  by  selecting  a  random  set  of  n 
new  values,  independently  with  replacement  from  ?  (note,  some 
values  will  be  repeated  once,  twice,  etc).  From  the  ordered  set 
of  the  Bootstrap  sample  the  10  percent  point  (j  ordered  value 
X(j)  oht*ln*d  consistent  with  the  90  percent  survival  number  is 
used  for  the  allowable  computation. 

N  Bootstrap  samples  are  obtained  with  corresponding  value 
(10  percent  points).  The  resulting  sequence  X3^j, 

x3(j,,...x",j,  estimates  the  distribution  of  the  10  percent  * 
point  of  F.  The  5  percent  point  of  the  Bootstrap  samples  results 


in  the  B  allowable  estimate  of  F.  N  usually  is  greater  than  500. 
If  sample  size  is  small,  preventing  adequate  representation  of 
percentile  values  relative  to  the  Bootstrap  process,  then  a 
smoothing  method  is  suggested.  Currently,  the  authors  are 
applying  the  2  parameter  weibull  tail  model  to  the  extreme 
quantile  region  of  the  empirical  distribution,  A  schematic  of 
this  procedure  is  shown  in  Figure  6.  Applying 
the  regression  model  as  shown  in  the  figure  will  provide 
generation  of  any'  sample  sise  including  extrapolation  to  lower 
quantile  values  (eg.  .01  for  A  allowable).  The  smoothing 
techniques  has  been  very  effective  in  smoothing  tail  region  data 
that  contains  contaminated  data  (eg.  outliers  and  bi-modality). 
The  flexibility  of  the  weibull  model  provides  a  good  tail 
representation. 


Weibull  Censored  Data  Allowable  Estimate  (Lawless) 

The  method  developed  by  [1]  for  obtaining  tolerance  bounds  for 
Weibull  extreme  quantile  values  was  applied  to  censored  data. 

The  data  was  right  censored  in  some  instances  by  80  percent  in 
order  to  preserve  a  homogenous  lower  extreme  quantile  region. 

This  scheme  is  extremely  effective  if  two  or  more  modes  existing 
in  the  data  and  only  data  from  lowest  mode  is  modeled 
with  remainder  of  data  censored  -  (See  Figure  7) .  The  B 
allowable  of  1.10  agrees  with  non-parametrlc  solution  while 
complete  sample  solution  of  1.06  is  too  conservative.  This 
technique  is  equally  effective  for  unimodal  data. 

Results  and  Discussion 

in  Figure  8a  the  complete  ranked  sample  is  displayed.  The 
allowable  estimates  for  the  functional  representation  are 
tabulated  at  right  of  figure.  The  Weibull,  normal  and  lognormal 
agree  quite  well  for  the  B  allowable  estimate  of  44.0  KSX.  The 
tail  region  is  not  adequately  represented  by  the  three  functions, 


therefore  reliance  on  the  44.0  KSI  value  could  result  in  an 
unsatisfactory  B  allowable  estimate.  The  third  ordered  number  of 
44.0  KSI  represents  90  percent  survivability  value  exclusive  of 
the  tolerance  bound.  This  indicates  an  obvious  overly 
non-conservative  allowable  estimate. 

t 

Figure  8b  displays  the  Weibull  censored  data  result  for  the 
complete  sample  shown  in  Figure  8a«  Within  the  figure  is 
tabulated  the  Design  B  38.8.  This  number  represents  the  B 
alllowahle  estimate  agreeing  with  the  conventional  non-parametric 
solution  of  38.5  which  is  the  first  ordered  value  when  sample 
sire  is  30.  The  reliability  of  the  non-parametric  solution  is 
usually  good  except  when  there  is  limited  amount  of  dispersion  in 
the  data.  In  this  case,  the  results  will  be  overly  conservative. 
The  12  percent  coefficient  of  variation  for  the  sample  in  Figure 
8a  is  acceptable,  therefore  the  non-parametiic  method  is 
acceptable  and  consequently  the  censored  data  result.  The  direct 
modeling  of  only  the  first  four  ordered  values  from  the  sample  of 
30  with  26  censored  number  has  provided  surprisingly  accurate 
allowable  estimates.  The  censored  data  method  has  consistently 
provided  accurate  results  for  most  data  sets  fox  arbitrary  tail 
sizes  greater  than  4. 

i 

The  ranked  data  of  18  values  is  displayed  in  Figure  9.  Mote, 
the  obvious  bi-modality  existing  in  the  data,  a  result  limiting 
selection  of  an  adequate  model  for  obtaining  the  allowable  value. 
The  results  from  conventional  functional  representation  Weibull, 
normal,  and  lognormal  are  43.0,  42.3,  and  42.8  respectively. 

These  values  appear  to  be  non-conservative  in  that  the  90  percent 
survival  value  is  43.0  not  including  reduction  due  to  the 
tolerance  estimate  related  to  the  sample  size.  The  obvious  poor 
representation  of  the  tail  region  has  resulted  in  this  error. 

in  the  tabulation  to  the  right  of  Figure  9,  the  results  of 
censored  data  solution,  Bootstrap  method  and  the  exponential 
model  are  shown  to  be  38.2,  36.1,  and  36.5  respectively.  The 
agreement  among  the  three  techniques  is  quite  good.  Only  the 
first  mode  (first  five  ordered  values)  were  modeled  except  for 
the  Bootstrap  process.  Finally,  the  non-parametric  (7)  result  is 


included  in  order  to  show  lowest  possible  allowable  value 
considering  all  models.  The  result  is  somewhat  conservative  due 
to  the  limited  sample  size.. 


Conclusions 


Three  methods  for  estimating  lower  confidence  bounds  on  extreme 
quantiles  have  been  discussed  in  this  paper:  an  exponential  tail 
estimator  developed  by  the  authors,  an  application  of  the 
Bootstrap  to  extreme  quantiles,  and  a  censored  data  method  due  to 
Lawless.  All  of  these  methods  are  based  on  the  idea  of 
estimating  confidence  on  an  extreme  quantile  from  a  tail  model, 
as  opposed  to  a  model  involving  the  complete  sample. 

Consequently,  all  three  are  potentially  useful  in  situations 
where  the  tail  appears  well  behaved  but  the  underlying 
distribution  is  in  doubt. 

The  method  of  Lawless  and  a  Bootstrap  procedure  were  introduced 
briefly  into  the  presentation  only  for  comparison  purposes. 
Lawless's  method  has  consistently  provided  good  results  in 
obtaining  the  allowable  values.  At  present,  considering  all 
available  models  it  is  the  "best".  The  primary  drawback  of  this 
censored  data  method  is  its  computational  complexity. 

Preliminary  results  from  the  Bootstrap  method  have  been  good  but 
inconclusive.  More  effort  is  needed  in  developing  this  method 
for  the  allowable  application. 

The  exponential  tail  estimator  was  presented  in  detail.  This 
new  estimator  for  confidence  on  extreme  quantiles  has  the 
advantage  of  simplicity  and  intuitive  appeal.  A  Monte  Carlo 
study  for  small  samples  from  Weibull  populations  revealed 
acceptable  bias  except  in  the  case  of  small  shape 
parameters.  Preliminary  investigation  of  tail  truncation  points 
suggests  that  modeling  a  longer  tail  reduces  expected  bias  and 
variance  in  the  case  of  small  Weibull  modulus. 

There  is  general  agreement  among  the  three  extreme  quantiles 
models  in  determining  allowable  estimates.  This  suggests  that 
these  models  can  provide  effective  alternatives  to  inadequate 
conventional  modele  where  model  uncertainty  exists. 

56 


r:? 

IV. 


References 

1.  Lawless,  o .  (1975)  "Construction  of  Tolerance  Bounds  for  the 
Extreme  Value  and  Weibull  Distributions  ",  Technometrics,  17, 
255-261 

2.  Efron,  B.  (1979)"  Bootstrap  Methods:  Another  Look  at  the 
Jackknife",  Annals  of  Statistics,  7,  1-26 

3.  Military  Standardization  Handbook,  Metallic  Materials  and 
Elements  for  Aerospace  Vehicle  Structures,  volumes  I  and  II, 
MIL-HDBK-5B,  15  August  1974,  Available  from  Superintendent  of 
Documents,  Government  Printing  Office,  Washington,  DC  20013,  USA 

4.  Breiman,  L.,  Stone  C.J.,  Gins,  J.D.  (1981),  "Further 
Developments  of  New  Methods  for  Estimating  Tail  probabilities  end 
Extreme  value  Distributions",  Final  Report  on  U.S.  Air  Force 
Office  of  Scientific  Research  Contract  No.  F49620-80-C-0037 , 
Technology  Services  Corporation,  Santa  Monl-s,  California 

5.  Lieblein,  J.  (1955),  "On  moments  of  order  Statistics  from  the 
Weibull  Distribution",  Annals  of  Mathematical  Statistics,  26, 
330-333 

6.  Hildebrand,  F.B.  (1974),  Numerical  Analysis,  McGraw-Hill:  New 
York 

7.  Billings,  G.B.,(1967)  "Improving  Binomial  Reliability 
Estimates  -  A  moderately  Distribution  Free  Technique  For  Small 
Sample  Reliability  Estimation",  Proceedings  of  the  Twelfth 
Conference  on  the  Design  of  Experiments  in  Army  Research, 
Development,  and  Testing  (ARO-D  Report  67-2} 


Acknowledgement 


To  Sharon  F.  DeMarco  of  AMMRC  for  her  excellent  work  in  the 
preparation  of  this  manuscript.  To  E.M.  Lenoe  of  AMMRC  for 
supporting  this  research  work. 


FIGURE  I 


Exact  'B*  Allowable,  Ectimated  Allowable  ar.d 
90%  Confidence  Interval  for  N«3Q 


Confidence 

Interval 


Smoothing  for  Bootstrap  Pr 


CENSORED  DATA  EVALUATION 


Figure  7  Wn  l  bul  1  Censored  Oats  Analysis 


STRESS  MT*  EUHU0T1O* 


rv 


H  *  •  T  ,  «  *  *  *  *  »  "  1  *  «  *  .  “  ■  "  *  *  H  "  *  "  -  "  |  *  «  *  »,'  */  •  ,*  •***,”  %”  V  'l*  "•*  *•*  '  **»*  '  »/  •,*  *,*  •,*  '  •  ‘  « ,*  f'  ,  *  •  "  '»/('  \ 


BASIC  Programs  for  Computing 
Reliability  and/or  Mean  Life 


Donald  W.  Rankin 
Lieutenant  Colonel 
US  Air  Force  (retired) 

1. _ Introduction.  In  essence,  these  BASIC  programs 

are  used  to  compute  those  definite  integrals  which  are 
associated  with  certain  prebab i *  ' 'ey  functions  and  which 
yield  confidence  levels.  A  f u '  :.'~matical  development 

of  the  formulae  used  will  be  fo  ir,  5  .  c  earlier  papers. 

[1]  and  [2] 

2^ _ Binomial  (Beta) _ distribution.  (Sampling  with 

replacement.)  If  L  denotes  the  level  of  confidence, 

r  the  unknown  probability  of  observing  a  success,  n 

the  sample  size  and  k  the  observed  number  of  failures, 

the  required  formula  is  ' 
z 

1-L  =  J"f(r)dr  where  f(r)  =  (n+1) C ( n , k ) rn~ k ( I- r ) k  . 
r=0 

Obviously,  z  falls  between  0  and  1. 

Program  1.  This  program  properly  is  used  to  test 
for  compliance  with  a  minimum  reliability  standard. 

Values  for  z,  n,  and  k  are  given,  from  which  L  is 
computed . 

Program  2.  This  program  determines  the  minimum 
reliability  associated  with  a  stated  confidence  level. 
Given  are  L,  n,  and  k,  from  which  z  is  computed. 

Trogram  3.  This  program  computes  a  "Best  Estimate" 
of  the  reliability  consistent  with  a  stated  confidence 
level.  It  minimizes  the  difference  z2~zi  between  the 
limits  of  integration  which  span  that  confidence  level. 
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3iSi _ Hypergeometr  1°  probability.  (Sampling  without 

replacement.)  If  L  denotes  the  level  of  confidence, 

N  the  population  size*  n  the  sample  size,  k  the 
number  of  observed  defectives  and  x  the  unknown  number 
of  defectives  in  the  original  population,  then  the  re¬ 
quired  formula  is 
x«m 

L  ■  5Z?(x)  where  P(x)  ■  £12 
xb  k 

It  is  found  that  m  falls  between  k  and  N-n+k. 

Program  4.  This  program  properly  is  used  to  test 
for  compliance  with  a  specified  maximum  allowable  number 
of  failures  in  the  original  population.  Values  for  N, 
n,  k  and  m  are  given,  from  which  L  is  computed. 

Program  5.  This  program  produces  a  listing  of  all 
the  discrete  probabilities  from  k  to  m.  The  upper 
bound  can  be  established  by  specifying  either  m  or  L. 

Program  6.  This  program  determines  the  maximum 
number  of  failures  in  the  original  population  which  is 
consistent  with  a  stated  confidence  level.  Given  are 
L,  N,  n  and  k,  from  which  m  is  computed. 

Program  7.  This  program  computes  a  "Best  Estimate" 
of  the  number  of  failures  in  the  original  population, 
consistent  with  a  stated  confidence  level.  All  of  the 
included  probabilities  exceed  every  excluded  one. 

Program  8.  This  program  lists  all  the  discrete 
probabilities  associated  with  the  "Best  Estimate"  of 
the  number  of  failures  in  the  original  population. 

(See  Program  7.) 

Program  9.  This  program  is  similar  to  Program  7, 
but  gives  in  addition  the  maximum  likelihood  estimate. 


,  k)  C(N-x.n-k) 
cTFPFTTnTT'r - * 


4.  Poisson  distribution.  (Constant  failure  rate.) 

■■■■»!  hi  n^i  ■  —  —  i  »i  m  ■  il  m  *  ■  — 

“Mean  Life"  is  given  by  the  reciprocal  of  the  constant 
(but  unknown)  failure  rate.  Let  T  denote  the  duration 
of  the  test  in  any  suitable  units,  and  k  the  number  of 
failures  observed  during  test  T.  If  x/T  denotes  the 
(unknown)  failure  rate  per  unit  in  the  whole  population, 
then  the  probability  that  x  does  not  exceed  some  value 
z  is  given  by 


P(«) 


Cl 

I 


f (x) dx  where  f (x)  ■  - 


in"" 


x*  0 

This  integral,  of  course,  is  equivalent  to  a  confidence 
level;  i.e.,  L  ■  P(z).  Once  z  has  been  determined, 
the  minimum  value  of  Mean  Life  is  given  by  T/z, 


Program  10.  This  program  properly  is  used  to  test 
for  compliance  with  a  specified  minimum  Mean  Life  in  the 
original  population.  Values  for  T  and  k  are  given 
and  T/z  is  specified,  from  which  L  is  computed. 


Program  11.  This  program  determines  the  minimum 
Mean  Life  which  is  consistent  with  a  stated  confidence 
level.  Given  are  L,  T  and  k,  from  which  z  (and 
hence  T/z)  is  computed. 


Program  12.  This  program  computes  a  "Best  Estimate" 
of  the  failure  rate  in  the  original  population  which  is 
consistent  with  a  stated  confidence  level.  Having  thus 
determined  and  z2  ,  the  corresponding  values  of 

Mean  Life  are  given  by  T/z2  and  T/z^  • 


5.  The  l.oeja  r  i  thro^of^  the_f  actor.i  ajU  The  presence 
of  large  factorials  in  the  several  formulae  virtually 
dictates  computation  by  logarithms.  The  BASIC  syntax 
normally  does  not  contain  a  routine  for  computing  the 
natural  logarithm  of  the  factorial  of  an  integer.  In 
view  of  this  deficiency,  a  representative  program  has 
been  included  which  achieves  the  desired  purpose.  It 
can  be  used  directly,  or  employed  as  a  model  when  com¬ 
posing  a  required  sub-routine.  For  single-digit  inte¬ 
gers,  it  computes  the  factorial  directly,  then  passes 
to  the  logarithm.  For  integers  of  two  or  more  digits, 
Stirling's  complete  formula  is  employed.  (See  [1],  p. 
278,)  The  result  is  accurate  to  11  or  12  significant 
digits,  or  to  the  accuracy  with  which  the  machine  com¬ 
putes  simpler  logarithms,  whichever  is  less.  It  must 
be  remembered  that  the  count  of  significant  digits  in¬ 
cludes  both  the  characteristic  and  the  mantissa  of  the 
logarithm.  But  the  subseouent  antilogarithm  will  con¬ 
tain  no  more  significant  digits  than  dees  the  mantissa 
of  the  logarithm. 
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APPENDIX  A 


BASIC  Program  Listings 


5  DISP  "The  LOGARITHM  'Of  the  FACTORIAL." 

10  DISP  "ENTER  Positive  Integer  (or  Zero)" 

15  INPUT  N 

20  FACTORIAL-1  @  LOGARITHM-O 

25  IF  N-0  THEN  80 

30  IF  N-l  THEN  80 

35  IF  N>9  THEN  65 

40  FOR  1-2  TO  N 

45  FACTORIAL-FACTORIAL* I 

50  NEXT  I 

55  LOGARITHM-LOG  (FACTORIAL) 

60  GOTO  80 

65  SERIES-((l/(7*N*2)-.5)/(30*N‘'2)  +  .5)/(6*N) 

70  STIRLING-LOG  (N) *( N+ . 5) -N+ (SERIES*. 918938533205) 

75  LOGARITHM -STIR LING 

80  PRINT  N j "  LOGARITHM  of  the  FACTORIAL  -  " ; LOGARITHM 
85  PRINT  CHR$  (10) 


2  PRINT  "  Program  l.",CHR$  (10) 

4  PRINT  "  This  program  tests  for  compliance  with" 

6  PRINT  "a  minimum  reliability  standard."  ,CHR$  (10) 

8  DISP  "ENTER  Sample  size"  0  INPUT  SAM 

10  DISP  "Enter  desired  reliability  standard"  £  INPUT  Z 

12  DISP  "Enter  number  of  failures  observed"  £  INPUT  K 

14  IP  K>1  THEN  22 

16  IP  K-l  THEN  20 

18  C0NF»1-Za( 1+SAM)  e  GOTO  48 

20  C0NF»1-Z*3AM*(1+SAM-Z*SAM)  @  GOTO  48 

22  P -1+SAM  6  GOSUE  62 

24  U-0  0  P-SAM-K  I  GOSUE  62 

26  V-Q  0  P-K  0  GOSUE  62 

28  FZ-EXP  ( K*LOG  (1-Z) +(SAM-K) *UOG  (Z)+U-V-0) 

30  IF  Z> 1-K/GAM  THEN  40  1  To  routine  for  larar  Z 

32  TZ-Z  *F  2/ ( 1 +SAM-K )  0  SZ-TZ  0  H-l 

34  TZ-Z  +  l’Z* (l+K-H)/( (1+SAM-K+H)* (1-2) )  8  S2-SZ+T2 

3C  H-l+F!  0  IP  TZ>.  00000000004  THEN  34 

38  CONF-l-SZ  £  GOTO  48  i  End  routine  fcr  small  Z 

40  T2- (1-Z)*FZ/(1+K)  0  SZ-TZ  $  H«1 

42  TZ* (1-Z)*TZ* (l+SAM-K-H)/((l+*+H)*Z)  0  SZ-3Z+TZ 

44  H-l+H  0  IP  TZ>. 00000000004  THEN  42 

46  CCNF-S2  1  End  routine  fcr  large  Z 

4fl  PCOu-CCMF+100000000  0  NFCO-FCON+FP  (PCON) 

50  CONF-IP  (NPCO)/ 100000000  I  Rounds  tc  8  places 
52  L'Fl'i'I  "Fcr  i  {•ample  rise  of "  ;  SAVj  "  item1: ,  er  org" 

54  t  r  I  :  'l  "which  exactly  '■■■ ;  "defect  iven  vere  observed," 

. .  :-r  j.  ■  ■  m  Cq]j' ability  exceed  ina" ;  z  ;"wil  1  occur  wit  l" 

00  '1,4*1  " probability  "iCD-PjCfiR?  (IOJiCKP?  (10) 

CO  ClOf 

'2  L-fCl  IP  cO1'  THE.*  68 

04  t*C=l  1  PO 1  =  1  n  F  1  FAC-F/CM  0  NEXT  I 
6  6  C* LOG  (FAC)  0  RETURN 

6  3  SCRI = ( ( 1 / ( 7*P "2 ) - . 5) / ( 30* P "2 )  +  . 5) / ( 6*P )  + . 9 139 38 53 320 5 

70  0=(i>+.5)*L OC  (P)-P+SFF1  0  RETURN 

7  2  K  N  0 


3  PRINT  "  Program  2.",CHR$  (10) 

6  PRINT  "  This  program  computes  a  minimum  reliability” 

9  PRINT  "consistent  with  a  specified  level  of  confidence.” 
12  PRINT  @  DISP  "ENTER  Sample  Size”  §  INPUT  SAM 
15  DISP  "Enter  desired  Level  of  Confidence"  @  INPUT  CONE 
18  DISP  "Enter  number  of  failures  observed”  @  INPUT  K 
21  Z-l-K/SAM  <a  IP  K >1  THEN  39 
24  IP  K-l  THEN  30 

27  Z« (1-CONF) a(1/(1+SAM) )  @  GOTO  66 
3  0  FZ«ZA  (SAM-1)  *S  AM*  ( 1+SAM)*  (1-Z.) 

33  SZ«ZASAM*  (  1+SAM-Z*S  AM)  @  DLZ- ( 1-CONP-S Z ) /F Z  @  Z-Z+DLZ 
36  IP  ABS  (DLZ) >.0000000004  THEN  30  ELSE  66 
39  P-l+SAM  @  GOSUB  87 
42  U-Q  @  P-SAM-K  @  GOSUB  87 
45  V-Q  @  P*K  @  GOSUB  87 

48  FZ-EXP  ( K*LOG  ( 1- Z )  +  ( SAM-K ) *  LOG  (Z)+U-V-Q) 

51  TZ-Z*FZ/( 1+S AM-K )  @  SZ-TZ  @  H-l 
54  TZ"Z*TZ*(l+K-H)/( ( 1+SAM-K+H) * (1-Z) )  @  SZ-SZ+TZ 
57  H-l+H  @  IF  TZ>. 00000000004  THEN  54  I  Integral  loop 
60  DLZ- (1-CONF-SZ) /FZ  @  Z-Z  +  DLZ 

63  IF  ABS  (DLZ) >.0000000004  THEN  48  1  Argument  loop 
66  XZ-Z*100000000  @  YZ-XZ+FP  (XZ)  @  RZ-IP  (YZ ) /10000000 0 
69  1  Rounds  Z  to  8  decimal  places 

72  PRINT  "For  a  sample  size  of "> SAM? " i terns ,  among" 

75  PRINT  "which  exactly" ; K > "defect ives  were  observed," 

78  PRINT  "a  reliability  exceeding" ;RZ; "will" 

81  PRINT  "occur  with  probability  ";COWF;CHR$  (10)>CHR$  (10) 
84  STOP 

87  LNFC :  IF  P>9  THEN  96 

90  FAC-1  @  FOR  I-l  TO  P  @  FAC-FAC*I  @  NEXT  I 
93  O-LOG  (FAC)  @  RETURN 

96  SERI* ( ( 1  /  ( 7*  P  A2 ) -. 5) / ( 3  0*  P A  2  ) +. 5) /( 6*P) +. 918938533205 
99  Q*  (P+ . 5 ) "LOG  (P ) -P+SERI  @  RETURN 
102  END 


75 


1  p  is  I N 1  "  Program.  3  .  "  ,  C  H  P  f  (]0) 

2  FFINT  "  This  program  computes  a  ";CHF$  (34); 

3  PFIN1  "Best  Es  tirrete"  ; C £J F $  (34) 

4  PFIMT  "cf  the  reliability  in  the  sense  cf  ir i r i r*um " 

5  pf  INI  "distance  between  bounds  for  a  spe.c  i  £  i  ed" 

6  PFINT  "level  cf  confidence ."; CEK$  (10) 

7  DISP  "ENTER  Sen  pie  Size"  @  INPUT  SAM 

G  DISP  "Enter  retired  confidence  level"  0  INPUT  CCN E 

9  DISP  "Enter  number  cf  failures  observed"  @  INPUT  K 

10  IE  K >  1  THEN  2  5 

11  It  K»  1  THEM  15 

12  PF  I  NT  CUP. »  ( 3  4)  ;‘"Iiee.t  Estimate’"  ;CKR$  (24.)  j 

13  PFIMT  "  is  net  defined  Per  zerc" 

14  PI  I  NT  "failures.  Use  Program  2  fer  reliability."  0  STOP 

15  A* 1-2/SAM 

16  FA*  A*  ( SAM-1 ) *SAM  *  ( 1  +  SAM.)  *  <  1-A.)  1  Login  A  lccp 

17  $  A  »  A " S  AM  *  ( 1  4S  A.N  -  A  *  S  A  M  )  0  7  *1  -F  A/  I  CAM*  ( 1  4  SAM.)  ) 

1£  F2.-1"  (SAM-J.)  *SAN*  (1.4SAM.)*  4  l.-Z )  1  Begin  2  lccp 

19  DLZ*3  I E  A/F‘  2- 1 ) *  Z*  1 1- l)  /  (  SAK-1- 2*S  AM.) 

20  2.=  24DLZ  @  IF  ALS  (DLL) > . OOOOOCOCOG4  THEN  If 

21  SZ*2"SAM* (14SAM-2*SAM)  k  Y-SAM-1/ ( 1 -2.) 

22  DLA*  (CONF-S24SA)  *Y/(  FA*  (2/A*  (SAM-1/  (1-A),)-Y.)  ) 

23  A*A  4  DLA  @  IF  AL<£  (DIA)  >SAM* ,  00000000001  THEN  16  ELSE  42 

24  1  Ere  cf  routine  fer  a  single  failure 

25  F*1.  +  SAN  0  GOSUD  52  l  Login  routine  fer  K  >  1 

26  U«Q  0  P*S  AM-'fl  p  COSUD  52 

27  V«Q  (?  P* K  0  GGSUB  52 

2 f  F*  1  - K/S AM  &  P2«SQK  (R*2-F* <1 -K/ (SAM-1) ) )  0  A-F-R5. 

29  FA“EXP  ( K*LOC  ( 1-A) 4 ( SAM- K) *LOG  <A)+0-V-C)  I  A  lccp 

30  2-P4PL  @  TA*A*FA/( 1+SAM-?)  0  SA*T A  Q  J“1 

21  TA*A*TA*  ( 1  +  K-J.)  /  (  (14SAM.-K+J.)  *  ll-A.)  )  0  SA-SA+TA 

22  0 *  1 4 o  0  IF  TA>K*. 00000000002  THEN  31  1  A  integral  lccp 
33  F  Z  =EXP  (  K*LOC  ( 1- 2)  +  ( SAM- 1: )  *  LOG  (Z.)+U-V-C) 

24  CLZ- IFA/FZ-1) * Z/ (SAM-K/ ( 1-Z ) )  ®  Z-Z4DLZ 

3  5  IF  AES  IDLE)  >I< *.  00000000002  THEN  22  1  F2»FA  lccp 

26  7 Z  - Z  *  F  2 /  ( 1 4  S  P  >•  -  /  )  (?  SZ  =  TZ  0  I>1 

27  TZ«Z*TZ*  (l+K-H)/(  (  1  +  S  AM-P4-H)  *  (  1-Z  )  )  0  SZ“S  Z4T  2 

36  H *  1 4 E  f  IF  TZ>X *.000000 00002  THEN  27  1  Z  integral  lccp 

39  K  =  S A M - K / (1-2 ) 

40  DLA*  lCOL-iF-SZ4SA)  *V/(  FA*  (Z/A*  (SAM-K/(1-A)  )-K)  )  0  A* A  +  DLA 

41  It  AES  (DLA)  >  £  A  M  *  ,  0  C  C  0  0  0  0  0  0  0 1  THEN  29  1  Er.d.cJ  A  lccp 

42  X  A  “  A  *  10  CCOOOOO  0  Y  A  *  X  A  4  F  P  (XA.)  Q  ZA*IP  (YA) /lOf’OOCOCO 

43  X2*Z*1DOCOOOOC  @  Y  Z  <*  X  2  4  F  P  (XZ)  Q  Z2-IP  <  YZ  )/l  OCCOOOOO 

44  1  Founds  results  to  8  decimal  places 

45  PFINT  "For  a  sample  size  cf" j SAM; " i terns ,  among" 

46  f  U  NT  "which  e;. ac 1 1  y’"  ; K;  "de f ec t  i ves  were  observed," 

4  7  FFINT  "the  "jCtir$  (34);'"Eest  Estimate" ;  CHE  ?  (34); 

41  p  PF  INI  "  c  f  the  reliability" 

4 c  PFINT  "falls  between  " ;  ZA ; "  and  ";Z2 

56  EFIM  "with  prctetil.ity  ;CONF;Ci!F$  (1C)|CHF$  (10) 

51  STOP 

52  LNFC:  IF  P>?  THEN  55  I  Logarithm  cf  the  factorial 
52  F  A C *  1  Q  For.  I  =  ]  TO  P  0  FAC*  E  AC*  I  C  NEXT  1 

54  C “LOG  (FAC)  0  FTTUIK 

6  5  SEFI-  .(J/(7*P‘2)-.5)/C'0*P-2)4  .')/(6*E)4.  r  ]  89  ?P  537205 

5  6  C  =  ( E  4  .  5 ) *LOC  (P) -P4FCF1  0  FTTUris 
5  7  FNC 


2  PFINT  "  Program  4.",CHR$  (10) 

4  PFINT  "  .This  program  tests  for  . compliance  with  a" 

6  PFINT  "specified  maximum  number  of  allowable  failures" 

P  PFINT  "in  the  original  population. " ,CHR$  (10) 

10  DISP  "ENTER  Population  Size"  @  INPUT  POP 
12  DIsftP-:" ENTER  Sample  Size"  0  INPUT  SAM 

14  DISP  "ENTER  Maximum  Number  of  Allowable  Failures"  @  INPUT  MXF 
16  DISP  "ENTER  Number  of  failures  observed"  @  INPUT  K 
18  IF  MXF  >*  K  THEN  22 

20  DISP  "**  ERROR  **  Maximum  already  exceeded"  0  GOTO  14 

22  P=l+POP  (B  GOSUB  60 

24  P-Q  @  P-l+SAM  0  GOSUB  60 

26  S*Q  0  P*1+MXF  l?  GOSUB  60 

2  8  T«Q  C“  P-POP-SAM  0  GOSUB  60 

30  U-Q  @  P.-POP-MXF  @  GOSUB  60 

32  V*Q  0  P-P0P-MXF-SAM4F  0  GOSUB  60 

34  K*Q  0  P-SAM-K  0  GOSUB  60 

26  X«Q  0  P«MXF-K  0  GOSUB  60 

.28  Y*0  0  P-l  +  K  @  GOSUB  60 

4 C  TX-lEXF  (U+V-P-K+S+T-X-Y-O)  0  SX*TX 

42  NU«TX* (14MXF-P) * (14SAM-P)  @  P«l+P 

44  TX»NU/(P*  IPOF-SAM-1-MXF+R).)  @  SX-SX4.TX 

46  IF  1X>K *.00000000002  THEN  42 

48  VX»SX*10CPPOOOG  0  YX-VX4FP  (VX)  0  ZX-IF  (YX)/3 00000000 
50  PFINT  "  The  population  consists  cf " ; FOP; " s in i lar  itemE,  A" 
52  PFINT  "ecirple  oi  size"  ;SAMj"is  drawn  which  contains  exactly"  j'F 
54  PFINT  "defectives.  The  original  pcpOlaticn  contained  fewer" 

56  PFINT  "then" ; 1+NXF; "defect ives  with  rrefcah i 1 1 ty" ; ZX 
58  DISP  SX  0  PFINT  CHF .?  (10)jC!IR$  flC)  0  STOP 
60  LNFCs  FAC* 1  0  IF  P>1  THEN  64 
62  C*0  0  RETURN 
64  IF  P> 5  THEN  70 

66  FOP  1-1  TO  P  0  FAC“FAC* I  0  NEXT  I 
C8  C'-LOO  if  AC)  0  RETURN 

7  0  SL.FI*  (  (l/(7*P',2)-.5)/(20*P',2)  +  .5)/(6*P)  +  . 91893853 32 05 
72  C* (P+ . 5) *LOG  (P) -P4SEF 1  a  RETURN 

74  END 


ft: 


PRINT  "  Program  5.",CHR$  (30) 

PRINT  "  This  program  has  an  expanded  printout  which" 

PRINT  "tabulates  both  specific  and  cumulative  probabilities" 

PRINT  "for  various  numbers  of  failures.  The  upper  bound  of" 

D  PRINT  "the  listing  .can  be  established  either  by  stating  the" 

2  PRINT  "itaximum  number  of  allowable  failures,  or  by  specify-" 

1  PRINT  "ing  a  desired  cumulative  probability. ";CHR$  (10) 

6  D1SP  "ENTER  Population  Size"  @  INPUT  POP 

8  DISP  "ENTER  Sample  Size"  9  INPUT  SAM 

0  DISP  "ENTER  Number  of  failures  observed"  @  INPUT  K@  A«0 

2  DISP  "ENTER  Desired  iupper  bound.  Use  integer  to  denote  maximum" 
4  DISP  "number  of  allowable  failures.  Or  use  decimal  fraction  td" 

6  DISP  "express  epprcpr late  .confidence  level." 

8  INPUT  MXF@  IP  MXF <1  THEN  38 
0  IF  FP  iMXF) *0  THEN  34 

2  DISP  "ERROR  ";MXF»"  Invalid  parameter "  j  G>  GOTO  22 
4  IF  MXF >«  K  THEN  38 

6  DISP  "**  ERROR  **  Maximum  already  exceeded"  0  GOTO  20 
8  Y»25  (?  DIM  B  (99)  ,C(99.) 

0  P-l+POP  9  GOSUB  82 
2  p«o  9  P-l+SAM  9  GOSUB  82 
4  S*Q  9  P*P0P-K  @  GOSUB  82 
6  T*Q  @  P-SAM-K  9  GOSUB  82 
8  B  »A)  *EXP  (T-R+S-Qi)  0C(A),*B(A) 

'  -  .  I  Mil  H  i  i  R  .  J  L _  •  il 


PRINT 

PRINT 


PFINT  "  The  population  consists  cf" t POP» " similar  items.  A" 
PRINT  "sample  of  size";SAM;"is  drawn  which  contains  exactly";* 
PRINT  "defectives.  The  various  probabilities  erej'";CHR$  (10) 
PFINT  "Number  of  Probability  Cumulative’" 

PRINT  "failures  of  occurrence  pr.ebebJli ty" ;CHR$  (10.) 

PFINT  USING  62  ;  K+A ; E ( A) ;C < A) 

IMAGE  XXDDD, 10X  ,  2 . 8D , 10X ,Z  . ED 

A*l  +  A  f)  IF  FP  <A/5).*0  THEN  PRINT 

p.  iA)-b(A-l>*  (A+K)*  (1+POP-SAM-A)/ (A*  rl+POP-K-A)  ) 

C  ( A)  *B  ( A. )  +C  (A-l )  9  IF  AO  Y  THEN  74 

DISP  "INSERT  UIV  PAGE.  Then  press  CONI"  9  PAUSL 

Y=Y+35  0  GOTO  56 

IF  NX F < 1  THEN  78 

IF  MXF  < A  +  K  THEN  PC  ELSE  6C 

IF  F XF>C (A-l )  THEN  60 

PUNT  CHR ?  (10)  jCHRv  (1C)  €  STOP 

LNFCs  E AC" 1  9  IF  P>1  THEN  86 

C=0  @  RETURN 

IF  P>9  THEN  97 

FOR  1=1  TO  P  0  FAC* FAC* 1  0  NEXT  I 
C*L0C  AC)  0  RETURN 

SEP  I  -  t  (1/(7*P*2)-.  5)  /  (.?0*P“2)  +  .5)  /  (6*P)  +  .T  1893  8  533205 

C*  (P+. 5) *L0G  (P)-P+SLRI  0  RETURN 

END 
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2  PRINT  "  Program  6.",CHR$  (JO) 

4  PRINT  "  This  program  computes  the  maximum  number  of'11 
6  PRINT  "defectives  in  the  original  population  ccrsistent" 

8  PRINT  "with  a  specified  confidence  level  and  with  the" 

10  PRINT  "observed  number  cf  sample  f allures. ";CHR$  (10) 

12  DISP  "ENTER  Population  Size"  @  INPUT  POP 

14  DISP  "ENTER  Sample  Size"  0  INPUT  SAM 

16  DISP  '"ENTER  Number  cf  failures  observed"  @  INPUT  K 

18  DISP  "ENTER  Desired  Confidence  Level.  (Use  decimal  fraction) 

20  INPUT  CONF@  Z-l-K/SAM  $  C-0  @  D-0  @  IF  K>1  THEN  32 

22  IF  K*1  THEN  26 

24  2«  il-CGNF.)*  (1/(1+6AM)  )  @  GOTO  52 
26  P2«Z'  (SAM-1)  ♦SAM*  (1+SAM)*  (1-Z,) 

28  SZ*Z*SAM*  (1+SAM-Z*SAM)  @  DL2- (1-CONF.-S2 ) /F Z  ?  2™Z40L2 

30  IF  ASS  (DLL) >.00000001  THEN  26  ELSE  52 

22  P-l+POP  @  GOSUD  94 

34  R«Q  6  P-l+SAM  (  GOSUB  94 

36  S*Q  0  P-POP-SAM  GOSUB  94 

28  U»Q  @  P-SAM-K  0  GOSUB  94 

40  X-Q  0  P-K  %  GOSUB  94 

42  FZ-EXP  (K*LOG  I 1-Z ) + (SAM-K) *L0G  (D+S-X-Q) 

4  4  TZ“2*F2'/  ( 1+SAM-K)  §  S2-TZ  %  H«1 

46  TZ*Z*TZ*  (l  +  K-H)/(  (1+SAN-K+R)  *  (1-Z.).)  @  SZ-JZ+TZ 
48  H-l+H  0  IF  TZ>. 00000001  THEN  46 

50  DLZ-  (1-CONF-SZ )  /F2  @  2-Z+DLZ  @  IF  ABS  (DLL)  >.  00000001  THEN  42 
52  MXF-lttT  (K-.5+(l-Z)* (l+POP-SAM) ) 

54  P-POP-MXF-SAM+K  0  GOSUL  94 

5  6  W-Q  @  P-POP.-MXF  (3  GOSUB  94 
50  V-Q  0  P*>  1 +MXF  C?  GOSUE  94 

6  0  T*0  0  P-RXF-K  @  COSUE  94 
62  Y»0  0  P *  1  +  K  9  GOSUD  94 

64  TX-EXP  (U  +\ -F-W+S+T-X-Y-1*.)  @  SX-TX 
66  NU-'iX*  (14NXF-P)*  (1+SAM-P)  0  P*l  +  f 
r.P  TX-NU/  (P*  (POP- SAM-l-MXF  +  P.))  0  sx*sx  +  ix 
70  IF  TX>K*.OOCCOf00002  THEN  66 

7  2  IF  SX>-  CONF  THEN'  76 

74  J-SX  fl  F-KXF  0  MXF-14HXF  0  C«S9  (8  IF  D«88  THEN  78  ELSE  54 
76  L=S  y '  0  G-MXF  0  MXF-MXF-1  0  D-68  0  IP  CO  99  THE  w  54 
70  XJ=J*1CC0PO0OO  (4  YJ-XJ4FP  (XJ)  0  JO-IP  ( YJ)/100000000 
00  X L=L* 1 CC C 0 0000  0  VL-XL+FP  (XL)  Q  LL* 1 1  (YL)/1C OCOOhflO 
02  PRINT  "  The  populat icr  ccrsists  cf";iOP;"£.ini)ar  iters,  A 
04  PFlN'l  "sample  ci  size" ;SAK;"is  drawn  which  contains  exactly";'!* 
66  print  "defectives.  The  original  population  contained  fewer" 
oF  PFINT  "than" >1+G; "defect  Ives  w  ith  p rohafc i 1 i ty" ; LL 
90  PFINT  "Or  fever  then'"  ;Gj  "with  probability"]  JJ 
92  PFINT  CHR$  (10) ]CHF$  (10)  @  STOP 
94  LNFC !  tAC-1  0  IF  P>1  THEN  98 
96  C=0  0  RETURN 

9  8  IF  P> 9  THEN  104 

100  FCP  1*1  TO  P  t  FAC -FAC* I  0  NEXT  I 
102  C-LOC  (FAC)  0  RETUFN 

104  SLI.l-  i  ( 1/  (  7  *P  *2  )  -  .  5)  /  (  2  0* P  *2  )  +  .  5)  /  ( 6*P )  +  .  9  1  8 536  5? 2205 
1 06  C* H  +.5)*LOG  (P)-P4SFFI  Q  FlTUFN 
106  FFL 


1  PRINT  "  Program  7.",CHR$  (10) 

2  PRINT  "  This  pregram  yields  e  '";CHR$  (34);  "Best  Estimate"; 

3  PRINT  CHRS  (34);"  cf" 

4  PRINT  "the  total  neither  cf  defectives  In  the  original'" 

5  PRINT  "population,  in  the  sense  of  minimum  distance" 

6  PRINT  "bet veer  bounds  for  a  specified  confidence  level," 

7  PRINT  CHR $  (10)  0  DISP  "ENTER  Population  size"  0  INPUT  POP 
P  DISP  "ENTER  Sample  size"  @  INPUT  5AM 

9  DISP  "Enter  number  cf  failures  observed"  0  INPUT  K 

10  DISP  "Enter  desired  confidence  level  (use  decimal  fraction) " 

11  INPUT  CCNf’0  IF  K >1  THEN  25 

12  IF  K*1  THEN  16 

13  PRINT  CHRS  (34); "Best  Estimate" ;CHP$  (34); 

14  print  "  is  net  defined  ;f c r  zero" 

15  PFIuT  "failures.  Use  Proqram  6  for  prebab J 1 i ty . "  @  GOTO  9  3 

16  A-l- 2/SAM 

17  FA«A * (SAM-1)  *SAM* (1+SAM)* (1-A)  @  SA»A"SAM* ( 1+SAM-A*SAM) 

IE  Z*1-FA/ ( SAM* (1+SAM) ) 

19  FZ»Z  * (SAM-1) *SAM* (1+SAM) *  ( 1-Z ) 

20  DLZ* |FA/F2-1)*Z*  <1-Z)/(SAM-1-Z*SAM) 

21  Z*Z+DLZ  @  IF  ADS  (DLZ) >. 000000001  THEN  19 

22  SZ»Z*8AM* ( 1+SAM-Z*SAM)  0  Y-SAM-1/ ( 1-Z) 

23  DLA*  (CGNF-SZ+SA)  *  Y/(  FA*  (Z/A*  »SAM-1/(J-A)  )-Y).) 

24  A-A+DLA  @  IF  ABS  (DLA) >. 000000001  THEN  17  ELSE  43 

25  P-1+P0P  t  GOSUC  8P 

26  R-C  0  P-l+SAM  0  GOSU6  88 

27  S*Q  0  P-POP-SAM  0  GOSUE  88 

2 8  U»Q  0  P-SAM-K  0  COSUB  8e 

29  X*Q  0  F.-R  0  GCSUB  EP 

30  F*0  0  FX=1-K/SAM  @  RZ-SQP  <RX*2-PX* ( 1-K/ ( SAM-1 ) ) )  @  A-PX-FZ 

31  PA-EXP  *K*LOG  ( 1-A)  +  (SAM-K )  *LCG  |A)+S.-X-Q.)  0  Z-RX+RZ 

32  TA*A*FA/(148AM-K)  0  SA*TA  0  0*1 

33  TA«A*TA* (l+F-J)/( (l+SAM-K+J) * (1-A) )  0  SA-SA+TA 

34  J=l  +  J  @  IF  'J  A>K*.  0000000002  THEN  23 

35  FZ=EXP  IK*LOG  ( 1- Z) + (SAM-K ) *LOG  «Z)+S-X-C) 

36  DLZ«  lPA/F7.-l)*Zy  (SAM-K/ (1-Z)  )  0  Z*2:+DLZ 
27  IF  ABS  (DLZ) >K* . OOOCOCCCC2  THEN  35 

2 L  TZ-Z*FZ'/(14SAM-K)  0  SZ-TZ  0  H-l 

39  TZ*Z*TZ*  (1+K-H) /(  (1  +  SAM-K+ll)  *  tl-Z)  )  0  SZ-SZ+TZ 

40  h-l+H  0  IF  1Z>I<*. 0000000002  THEN  39 

41  WX-SAK-R/(l-£)  0  DLA*  (CONF-SZ+SA)  *KX/  (FA*  (Z/A*  (SAM-K/  ( 1-A  )  )  -WX)  ) 

42  A»A+DLA  0  IF  ABS  ( DLA) >SAM* . 00000000 01  THEN  31 
4.2  MNF-1N1  (K- .  5+  ( 1-  2)  *  ( 1  +  POE-SAM).) 

44  MXF-IN1  ( K+. 5+ ( 1-A) * ( 1+POP-SAK) ) 

45  PRINT  "  The  population  consists  cf";LOP; 

46  PFIN'l  "similar  Jteirs." 

47  PRINT  "A  sample  cf  e  i  ze‘" ;  SAM ; "  i  s  drawn  which  certains  exactly" 
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W  P  PI  NT  "A  sarnie  cf  size" ;SAM;"is  drawn  which  contains  exactly11 
48  P*P0P-MXF-SAM4K  @  GOSUE  88 

4  9  tv-Q  @  E«FOP.-MXF  @  GGSUB  88 

5  0  V=0  0  P.*MXF  @  GOSUE  88 

51  T-Q  @  P*MXF-K  @  GOSUE  88 

52  Y«Q  @  P«P0P.-MNF-SAM4K  @  GOSUEf  88 

53  N=Q  @  P*POP-MNF  @  GOSUE  88 

54  M*Q  @  P*MNF  0  GOSUE  88 

55  L*Q  @  P*MNF-K  @  GOSUE  88 

56  G«C  0  F-l+K 

57  PX-EXP  (U4V-F-W4S4T-X-Y-F) 

58  TX=PX*  ll4MXF.)/P  @  SX-TX 

59  NU«TX*  (14MXF-P.)  *  ( 1+SAM-P)  /  (P0P-SAM-MXF4P) 

60  P«14P  @  TX»N U/P  @  SX-SX4TX 

61  IP  TX>K* . 00C0D000002  THEN  59 

62  CX-SX  0  CF-MXP  @  P-14K 

6  3  PY*EX P  (U4M-F-N4S4L-X-G-F.) 

64  TY*PY* (14MNF )/P  @  SY-TY 

65  MU*TY*  { 14MNF-P.)  *  ( 14SAM-P.)  /  ( POF-SAM-MNF 4P) 

6  6  P*14P  (3  TY*MU/P  @  SY-SY4TY 
67  IF  ,'JY>K*.  00000000002  THEN  65 

66  CY=SY  @  CE=MNF  @  P-14K 

69  D1SP  MNF ,MXF 

70  IF  CX4PY-CY<C0NF  THEN  78 

71  IF  PX<PY  THEN  75 

7  2  IF  CX-CY <CONF  THEN  8  3 

73  MNF-MNF41  @  PY»PY*NNF*  i  POE-SAM-MNF4  K4l)  /  (  (MNF-K )  *  ( 14P0P-MNF )  ) 

74  CY-CY4PY  @  GOTO  69 

75  IF  CX4PY-PX-CY <CONF  THEN  83 

76  MXP*NXF-1  @  CX^CX-PX 

7  7  PX=PX*  ll-K/(]4WXF))*(P0P-N!XF)y(P0F-MXF-SAM4K)  @  GOTO  69 

78  IF  PX< PY  THEN  81 

79  MXF-MXF41  @  PX*PX *JSXF*  IP0E-SAM-NXF4K41 ) /  <  (i*XF-K)  *  (14P0P-MXF)  ) 

80  CX-CX4PX  0  GOTO  69 

81  MNF^MNP-l  0  CY*CY-PY 

82  PY-PY* ( 1-K/ ( 14MNF ) ) * (POP-MNF }/ ( P0P-MNF-SAM4K)  0  GOTO  69 

83  XX*  (CX4PY-CY)*100000000  0  YY-XX4FP  (XX)  0  Z « J P  ( YY)/100000000 

84  PRINT  X;  "defectives.  At  the"  ,*  Z Z  j  "level  of  confi-" 

85  POINT  "dcnce,  the  original  population  contained  between" 

86  PFI  NT  MNF;  "and"  jF'XF;  "defect  i  ves,  inclusive." 

87  PFINT  CHP  $  (10)  @  STOP 

88  LNFC :  IF  P>9  THEN  91 

89  FAOl  @  FOP  1*1  TO  P  @  FAC-FAC*!  0  NEXT  I 

90  Q»LOC  i FAC)  0  PETUPN 

91  SEPI-  i  (1/(7*P',2)-. 5)  / (.  30  *PA2)4,5)/(6*P)4. 916938533205 

92  C*(P4.5)*LOG  ( P)  -P4SERI  0  FFTUFN 

93  PRINT  CHR$  (10)jCHF$  (10)  0  END 
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2  PRINT  "  Program  8.",CHR$  (.10) 

4  PRINT  "  This  program  has  an  expanded  printout  which” 

6  PRINT  "tabulates  both  specific  and  cumulative  probabilities" 

8  PFINT  "for  various  numbers  cf  failures.  The  upper  and  lower'" 

10  PFINT  "bounds  cf  the  listing  reflect  the  maximum  end  minimum" 

12  PFINT  "number  of  failures  associated  with  the'">CHR$  (34)  ; 

14  PRINT  "Best  Estimate" ;CHR$  (34) 

16  PRINT  "consistent  with  a  stated  level  of  confidence. " ;CHR$  (10) 
18  DISP  " ENTFR  Population  Si2e"  @  INPUT  POP 
20  DISP  "ENTFR  Sample  Size"  @  INPUT  SAM 

22  DISP  "ENTER  Number  cf  failures  observed"  @  INPUT  A»0 
24  DISP  "ENTER  Confidence  level.  (Use  decimal  fraction.)" 

26  INPUT  CONF0  DIM  P(99),C(99) 

26  IF  P0P-SAM>f  9  THEN  98 
30  P-l+POP  0  GOSUD  84 
32  R-Q  0  P-l+SAM  0  GOSUB  84 

3  4  S-C  0  P-POF.-K  0  GOSUB  84 
36  T*Q  0  P-SAM-K  0  GOSUB  84 

38  P  I  A)  -EXP  (T-R4S-Q )  0  C(A),«P(A) 

40  FOF  A-l  TO  POP-SAM 

42  P  (A)  «P  IA-1)  *  (K+A)  *  (l+POP-A-SAM)'/(  A*  il+POP-K-A)  ) 

44  C  (A),-P  (A)  +C  (A-l)  @  NEXT  A 

46  PFINT  "  The  population  consists  cf 11 ;  POP; 11  similar  items." 

48  PRINT  "A  sample  of  size" ;SAM j " i s  drawn  which  contains  exactly" 
50  PFINT  K; "defectives.  The  various  pr.obab i  1  i ties  ere^jCHRS  (10) 
52  PRINT  "Number  of  Probability  Cumulative" 

54  PFINT  "failures  of  occurrence  pr.obahJlity";CHR$  (10) 

56  B-0  0  D“POP-SAM 

58  IF  P(B) < P ( D )  THEN  64 

60  IF  C  (D-l) -C  (B.)  4P  (U)  <CONF  THEN  6e 

62  D-D-l  0  GOTO  58 

64  IF  C(D)-C(E) <CONF  THEN  68 

66  B-134-1  0  GOTO  58 

68  A-0  0  FCF  H-B  TO  D 

70  PRINT  USING  72  >  K4H; P (H) )C (H) 

72  IMAGE  XXDDD , ] OX ,Z .BD,10X,Z.8D 
74  A-14-A  @  IF  FP  (A/5 ) *0  THEN  PRINT 
76  NEXT  H  0  PFINT 

78  PFINT  USING  80  ;  "Sum" ,C (D) -C (B-l) , " (Ccnf .  level)" 

80  IMAGE  XXAAA,10Xf Z.8D  ,7X,13A 
82  PRINT  CHR$  (10);CHR$  (10)  0  STOP 
84  LNFCs  FAC-1  0  IF  P>1  THEN  88 
86  0-0  0  RETURN 
88  IF  P>9  THEN  94 

90  FOR  1-1  TO  P  @  FAC-FAC* I  @  NEXT  1 

9  2  C-LOG  I  FAC)  @  RETURN 

94  S ERI-  (  il/  (7*P"2)-».  5)/ (30*P *2)+  .5)/  (6*P)  +  . 918938533205 
9  6  Q-  (P4 , 5)  *LOG  (  P)  -P4-S ER I  0  RETURN 

98  DISP  "**  ERROR  **  Population  too  large  fer  arre)." 

100  DISP  "  Redimension  on  lines  26  end  2P." 

102  END 


1  print  "  Program  9.",CHR$  (10) 

2  PRINT  "  This  program  yields  e  '"?CHR$  (34) ? 

3  PRINT  "Best  Eat imate"  ?CHR$  (34.)?"  of" 

4  PFINT  "the  total  number  of  defectives  in  the  original 

5  PRINT  "population,  in  the  ,sense  of  minimum  distance" 

S  print  "between  bounds  for  a  specified  confidence  level." 

7  PRINT  "Also  given  is  the  maximum  likelihood  estimate. 

B  PRINT  9  DISP  "ENTER  Population  Size"  §  INPUT  POP 

9  DISP  "ENTER  Sample  Size"  @  INPUT  SAM 

10  DISP  "Enter  number  of  failures  observed"  @  INPUT  K 

11  DISP  "Enter  desired  confidence  level  (use  decimal  fraction) 

12  INPUT  CONF@  IF  K>1  THEN  26 

13  IF  K*1  THEN  17 

14  PRINT  CHF$  (34)?"Best  Estimate" ?CHR$  (34); 

15  PRINT  "  is  not  defined  ifcr  zerd" 

16  PRINT  "failures.  Use  Program  6  for  probability."  @  GOTO  98 

17  A*l-2/SAM 

18  FA*A*  (SAM-1)  *SAM*  (1+SAM)*  (1-A)  @  SA-A  SAM*  ( 1+SAM-A*SAM.) 

19  2*1 -FA'/  ( SAM*  ( 1+SAM).) 

20  FZ»2*  (SAM-1)  *SAM*  (1+SAM)*  (1-Z) 

21  DLZ“ |FA/FZ“1)*Z*( 1-Z) / ( SAM-l-Z* 5AM) 

22  Z-Z+DLZ  @  IF  ABS  (DLZ) >.000000001  THEN  20 

23  SZ-Z^SAM* (1+SAM-Z*SAM)  @  V-SAM-1/ ( 1-Z) 

24  DLA= ICONF-SZ+SA) *Y/(FA* IZ/A* ( SAM-1/ ( 1-A) ) -Y) ) 

25  A-A+DLA  @  IF  ABS  (DLA) >. 000000001  THEN  18  ELSE  44 

26  P-l+POP  @  GOSUB  93 

2  7  R“Q  9  E* 1+SAM  9  GOSUB  93 
2  8  S*0  @  P*  POP, -SAM  9  GOSUB  93 

29  U*C  9  P*SAM-K  9  GOSUB  93 

30  X«C  @  P“K  @  GOSUB  93 

31  F»C  @  RX«1-K/SAM  @  RZ-SQP  (  FX*2-RX*  ( ,1 -K/ ( SAM-1)  )  )  @  A-FX-PZ 

32  FA*EXP  tK*LOG  ( 1-A)  +  ( SAM-K) *  LOG  |A)+S-X-Q.)  @  Z-RX+RZ 

33  1 A*A*FA/ ( 14SAM-K)  9  SA*TA  @  J»1 

34  TA-A*TA* (1+K-J)/ ( (1+SAM-K+J) * » 1-A) )  9  SA-SA+TA 
■J5  J*l  +  J  @  IF  ,TA>K* .  000000000  2  THEN  34 

36  FZ®  EXP  «K*LOG  ( 1-Z) +  { SAM-K)  *  LOG  IZ)+S-X-Q) 

37  DLZ*  lFA/FZ-1)  *  2/  (SAM-K/  (1-Z),)  @  Z-Z+DLZ 
36  IF  ABS  (DLZ ) >K*. 0000000002  THEN  36 

39  TZ»2*F2/ (l+SAM-K)  <?  S2-TZ  9  H-l 

40  T2*2*TZ* ( 1+K-H) / ( ( 1+SAM-K+H) *  II- Z) )  @  SZ«SZ+TZ 

41  H * 1 +H  9  IF  TZ>K*. 0000000002  THEN  40 

42  WX-SAM-K/(1-Z.)  9  DLA-  (CONF-SZ+SA)  *WX/ ( FA*  (Z/A*  (SAM-K/ ( 1-A),) -WX)  ) 
4.3  A*A+DLA  9  IF  ABS  (DLA)  >SAM*  .0000000001  THEN  32 

44  MNF* INT  ( K- . 5+( 1- Z) * ( 1+POP-SAM) ) 

45  MXF* INT  (K+, 5+(l-A)* (1+POP-SAM) )  ^  „  ,  ..  „ 

46  PFINT  "  The  population  consiets  of * > POP ? " f in i lar  items. 

47  PRINT  "a  sample  of  e  J  ze’"  ?  SAM  ? "  i  s  drawn  which  contains  exactly 

48  P-POP-MXF-SAM+K  9  GOSUE  93 

49  W»Q  p-FOP-MXF  9  GOSUB  93 

50  V«Q  @  P*MXF  @  GOSUB  93 


50  V-Q  @  P-MXF  @  GOSUB  93 

51  T-0  @  P-MXF-K  @  GOSUB  93 

52  Y-Q  g  P-POP-MNF-SAM+K  @  GOSUB  93 
5.3  N-Q  @  P-POP-MNF  @  GOSUB  93 

54  M-Q  @  P-MNP  @  GOSUB  93 

55  L«Q  @  P-MNF-K  @  GOSUB  93 

56  G-Q  @  P-l+K  <?  PX-EXP  (U+V-R-W+S+T-X-Y-F) 

57  TX-PX*  ll+MXF)/P  @  SX-TX 

58  NU^TX*  (1+MXF-R)*  (1+SAM-P.) / (POP-SAM-MXF+P.) 

59  P-l+P  @  TX-NU/P  @  SX-SX+.TX 

60  IF  >TX>K* . 00000000002  THEN  58 

61  CX-SX  @  CF*MXF  @  P-l+K  @  PY-EXP  (U+M-R-N+S+L-X-G-F.) 

62  TY-PY*  ll+MNF,)/P  @  SY-TY 

63  MU-TY*  (1+MNF-P.)  *  ( 1+SAM-P.)  /  (POP-SAM-MNF+P.) 

64  P«l+P  @  TY-MU/P  @  S  Y-SY+.TY 

65  IF  1Y>K*. 00000000002  THEN  63 

66  CY-SY  @  CE-MNF  @  P-l+K 

67  D1SP  MNF,MXF 

68  IF  CX+PY-C Y  <CONF  THEN  76 

69  IF  PX<PY  THEN  73 

70  IF  CX-CY<CONF  THEN  81 

71  MNF-MNF+1  @  PY-PY*MNF*  (POE-SAM-MNF+K+1)'/ (  (MNF-K)*  (1+POP-MNF)  ) 

72  CY-CY+PY  @  GOTO  67 

73  IF  CX+PY-PX-CY<CONF  THEN  81 

74  MXF-MXF-1  @  CX-CX-PX 

75  PX-PX*  (1-K/(1+MXF.),)*  (P0P-WXF)7(P0P-MXF-SAM+K)  @  GOTO  67 

76  IF  PX<PY  THEN  79 

77  MXF-MXF+1  @  PX-PX*MXF*  (POP,~SAM“MXF+K  +  l)’/  (  (MXF-K.)  +  (1+POP-MXF)  ) 

78  CX-CX+PX  §  GOTO  67 

79  MNF-MNF-1  @  CY-CY-PY 

80  PY.-PY*  (L-K/  ( 1+NNF  ).)  *  (POF-MNF.)'/  (POP-MNF-SAM+K)  @  GOTO  67 

81  XX- (CX+PY-CY)*100000000  @  YY-XX+FP  (XX)  @  ZZ-IP  ( YY.) /1000 00000 

82  PFINI  Kj "defectives.  At  the" ;ZZ;"level  of  confi-’" 

83  print  "dence,  the  crlgjnal  population  contained  between" 

84  PRINT  MNF;"and" ;MXF; "defectives,  lnclueive. " >CHR$  (10) 

85  MLX-  1 1 4POP.)  *K/SAM 

86  MLY-INT  (MLX)  @  IF  MLX-MLY  THEN  89 

87  PPIN'I  "The  iraxlrouir  likelihccd  estiirate  j.e"  ;MLY;  "defect  1  ves.  " 

88  GOTO  98 

89  PRINT  "There  ere  two  equally  likely  iraxlmum  likelihccd" 

90  PRIM  "estimates.  They  are'";MLX-l;"and";MLXj"defectlves." 

91  GOTO  98 

92  STOP 

93  LNFC :  IF  P>9  THEN  96 

94  FAC-1  @  FOF  1*1  TO  P  @  FAC-FAC*I  @  NEXT  I 

95  Q-LOG  (FAC)  @  RETURN 

96  SERI-  I  <1/(7*P*2)-.  5)/(.30*P*?)  +  .5)/(6*P)+.  918938533205 

97  Q«lP+.5)*LOG  (P )  -P+SERI  #  RETURN 

98  PFINT  CHR $  (10)jCHR?  (10)  @  END 


3  PRINT  "  Program  10.",CHR$  (10) 

6  PRINT  "  This  program  tests  for  compliance  with  e" 

9  PRINT  "minimum  mean  life  standard ." ,CHR$  (10) 

12  DISP  "ENTER  Duration  of  Test,  UNITS  Employed." 

15  DISP  "  (Use  comma  between.)" 

18  INPUT  DUR ,UNIT$ 

21  DISP  "Enter  desired  mean  life  standard  in  ";UNIT$ 

24  INPUT  MLS(?  DISP  "Enter  number  of  failures  observed" 

27  INPUT  K@  Z-DUR/MLS  9  IP  K>0  THEN  33 
30  CONF- 1-EXP  (-Z)  0  GOTO  72 
33  IP  K >9  THEN  54 

36  FAC-1  9  FOR  1-2  TO  K  @  FAC -FAC* I  9  NEXT  I 
39  F Z- Z * K/ ( FAC *E X P  (Z))  Q  TZ-K*FZ/Z 
42  S2-FZ+TZ  9  IF  K-l  THEN  51 
45  FOR  J-l  TO  K  0  TZ-TZ* (K-J) /Z 
48  SZ-SZ+TZ  9  NEXT  J 
51  CONF-l-SZ  9  GOTO  72 

5  4  SERI-((l/(7*K'‘2)-.5)/(30*K*2)4-.5)/(6*K) 

57  FZ-EXP  (K* (1 +LOG  (Z) )- (K+ .5) *L0G  ( K) -Z- (SBRI+ .9189385332) ) 
60  TZ-Z *FZ/ (K+l )  @  CONF-TZ  9  L-2 
63  TZ-Z *TZ/ ( K+L )  @  CONF-CONF+TZ 
66  IF  TZ<. 00000000001  THEN  72 
69  L-l+L  @  GOTO  63 

72  XX -CONF* 100000000  9  YY-XX+FP  (XX)  §  ZZ-IP  ( YY) /100000000 
75  PRINT  "For  a  test  duration  of  ">DUR;"  ";UNIT$ 

78  PRINT  "during  which  exactly  ">K;"  failures  were  observed, 
81  PRINT  "a  mean  life  exceeding  ";MLSj"  ";UNIT$ 

84  PRINT  "will  occur  with  probability  "fZZfCHR$  (10) 

87  END 


3  PRINT  "  Program  ]].",CHR$  (10) 

6  PRIN1  "  This  program  computes  a  minimum  mean  life" 

9  PRINT  "consistent  with  a  specified  leve]  cf  corf  idence.'"  ,CHR$  (10) 

12  DISP  "ENTER  Duration  cf  Test,  UNITS  Employed.  (U6e  comma  between.) 
15  INPUT  DUF, UNITS 

18  DISP  "Enter  desired  Level  of  Confidence.  (Uee  decimal  fraction.^ 
21  INPUT  CONE®  DISP  '"Enter  number  cf  failures  .observed" 

24  INPUT  K0  IF  K>0  THEN  30 

27  Z— LOG  ll-CONF)  0  GOTO  75 

30  Z-K+SQF  (K)  @  IF  K>9  THEN  54 

33  FAC-1  0  FOR  1-2  TO  K  @  FAC-FAC*!  0  NEXT  I 

36  FZ-ZaK/(FAC*EXP  (Z).)  §  TZ»K*FZ/Z 

39  SZ-FZ+TZ  0  IF  K-l  THEN  48 

42  FOR  J-l  TO  K  @  TZ-TZ* (K-J)/Z 

45  SZ-SZ+TZ  0  NEXT  J 

48  DLZ-Z*  ( 1-SQR  (1-2*  (  2-K)  *  (CONF.-l+SZ)i/(Z*FZ)  ).)/(Z-K) 

51  Z-Z+DLZ  @  IF  ABS  (DLZ) <K*2* .0000000001  THEN  75  ELSE  36 
54  SERI-( (l/(7*KA2)-.5)/(30*K*2)+.5)/(6*K) 

57  FZ-EXP  <K*  ( 1+LOG  (Z).).-( K+.  5  )  *LOG  (K)-Z- (SEFI  +  .  9189385332)  ) 

60  TZ«Z*F2/(K+1)  0  TCON-TZ  @  L-2 

63  TZ-Z*TZ/(K+L)  @  TCON-TCON+TZ  0  IF  TZ/ . 00000000001  THEN  69 
66  L-L+l  @  GOTO  63 

69  DLZ-Z*  (1-SOR  (1-2*  (Z-K)*  (CONF-TCON)//(Z*FZ).)  )/(Z-K) 

72  Z-Z+DLZ  @  IF  ABS  (DLZ)<K*2* .0000000001  THEN  75  ELSE  57 
75  XX-100000*DUR/Z  0  YY-XX+FP  (XX)  0  MLS-IP  (YY)/300000 
78  PRINT  "For  a  test  duration  of  "»DUP>"  ";UNIT$ 

81  PRINT  "during  which  exactly  K>"  failures  were  observed," 

84  PRINT  "a  mean  life  exceeding  "jMLSj’"  ";UNIT$ 

87  PRINT  "will  occur  wi th  >probebili ty  ";CONFfCHF$  (10) 

90  PRINT  CHR$  (10) 

93  END 


2  PRINT 
4  PRINT 
6  PRINT 
8  PRINT 


"  Program  32.",CHR$  (10) 

"  This  prccrair  computeb  the  ";CHRf  (34); 

"Best  Estimate"  ;CHF$  (34.);"  cf" 

"the  Mean  Life  consistent  with  a  specified  Level 


10 
12 
14 
16 
16 
20 
22 
24 
26 
28 
30 
32 
34 
36 
38 
40 
42 
44 
46 
48 
50 
52 
54 
56 
58 
60 
62 
64 
66 
68 
70 
72 
74 
76 
78 
00 
82 
84 
86 
88 
90 
92 
94 
96 
98 
100 
102 
104 
106 
108 
110 


(Use  comma  between.) 


PRINT  "of  Corf iderce.‘",CHR$  ( 1 Q) 

DISP  "ENTER  Duration  cf  Test,  UNITS  employed. 

DUR , UNIT$  ,  ,  ,  _ 

"ENTER  des ired  Level  of  Confidence.  (Use  decimal  fraction.!) 
CONF9  DISP  "ENTER  number  cf  failures  .observed," 

Kg  IF  K>0  THEN  28 

CHR$  (34) ;"Best  Estimate" ;CHR$  (34); 

"  is  net  defined  for  2ero  failures." 

"Uae  Prcaram  11  to  compute  probability."  9  GOTO 
IF  K>1  THEN  44 
(2)  9  A-FZ 


INPUT 
DISP 
INPUT 
INPUT 
PRINT 
PRINT 
PRINT 
Z-2  9 
FZ 
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Z/EXP  . 

FA-A/EXP  (A)  9  DLA-(P2-FA)*A/(FA*  ll-A) ) 

A-A+DLA  9  ALB-A  9  IF  ABS  (DLA) >K*. 00 00 000005  THEN  32 
TCON-FZ/A-F2/2 

DLZ-  IZ-  (2-1).)  *  (1-SOF  (1-2*  (2-1)  *  (1-A)*  (CONF-TCON)  /  (  ( 2-A)  *FZ).)  ) 
2-Z+DLZ  @  F2-Z/EXP  (2) 

IF  ABS  (DLZ) >KA2*. 0000000001  THEN  30  ELSE  88 
Z-K+SOF  (K)  9  IF  K>9  THEN  66 

A-K-SQR  IK)  @  FAC-1  @  FOR  1*2  TO  K  9  FAC-FAC*I  §  NEXT  I 
FZ»ZaK/(FAC*EXP  (Z),)  9  TZ-K*FZ/Z  @  S2-FZ+TZ 
FOF  J«1  TO  K  9  TZ-TZ*  (K-JJ/Z  0  SZ-SZ+T2  P  NEXT  J 
FA-A*K/(FAC*EXP  (A).)  9  DLA-A  *  (FZ/FA-3 )  /  ( K-A) 

A-A+DLA  9  ALB-A  @  IF  ABS  (DLA) >K* .0000000005  THEN  52 
TA-K*FZ/A  9  SA-FZ+TA 

FOP  H-l  TO  K  0  TA-TA* <K-H)/A  9  SA-SA+TA  P  NEXT  H 
TCON-SA-S Z 

DLZ-Z*  (1-SOR  (1-2M2-K)*  (K-A)*  (CCNF-TCON)  /  ( K*  (Z-A)  *FZ).)  )  /  ( Z-R) 
Z-Z+DLZ  @  IF  ABS  (DLZ)>K*2*. 00 00000001  THEN  48  ELSE  88 
SERI-((l/(7*K,'2)-.5)/(30*K*2)  +  .5)/(6*K)  0A-K-SQR  (K) 

LNFC» ( K+ . 5 ) *  LOG  (K)-K+ ( SERI  +  . 91893853 3205) 

FZ-EXP  (K*LOG  (Z)-Z-LNFC)  9  TZ-K*FZ/Z  @  SZ-FZ+TZ  9  O-K-l 
TZ-TZ*C/Z  9  SZ-SZ+T2  9  Q-Q-l  9  IF  TZ>, 00000000004  THEN  72 
FA-EXP  •!(* LOG  (A)  -A-LNFC )  §  DLA-A*  (FZ/FA-1)/  (K-A)  0  A-A+DLA 
IF  ABS  (DLA) >K* .0000000005  THEN  74 
SA-0  9  TA-FA  9  Q-K+l 

TA»A*TA/0  9  SA-SA+TA  9  0-0+1  9  IF  .1A> ,  00000000004  THEN  80 
TCON-l-SZ-SA 

DLZ-2*  ll-SQR  (1-2*  (Z-K) * (K-A)* (CONF-TCON) /( K* (Z-A) *FZ) ) )/(Z-K) 
Z-Z+DLZ  0  IF  ABS  (DLZ)  >^2*  .0000000001  THEN  70 
Q0-100000*DUR  9  XX-OO/Z  9  YY-XX+FP  iXX)  9  ZZ-IP  (YY)/100000 
CC-OO/A  0  BB-CC+FP  (CC)  0  AA-IP  »BB)/100000  0  HH-OC/K 
JJ-hH+FP  (HH)  9  KK-IP  i.JJ)/100000 
PP1N1  "  For  a  test  duration  of " ;DUR;UNIT$ 

PRINT  "exactlyr";K;"feilures  were  observed." 

PRINT  CHR?  (10) ;"At  the  ";CONF;"  Level  of  Confidence,  the 
PRINT  CHF$  (34) ; "Best  Estimate"; 

PRINT  CHP$  (34);"  cf  the  Fean  Life  falls" 

PRINT  "between" ; ZZj"  and" 

PRINT  Afl ; UNITS; "  .  The  iranjirun  like-" 

PRINT  "lihccc  estimate  is  "  ;KK;UNIT$; " 

PRINT  @  END 


APPENDIX  B 


The  Programs  Exercised 


Program  1 . 

This  program  tests  for  compliance  with 
a  minimum  reliability  standard. 

For  a  sample  size  of  7  items,  among 
which  exactly  2  defectives  were  observed, 
a  reliability  exceeding  .7  will  occur  with 
probability  .44822619 


Program  1. 

This  program  tests  for  compliance  with 
a  minimum  reliability  standard. 

For  a  sample  size  of  21  items,  among 
which  exactly  6  defectives  were  observed, 
a  reliability  exceeding  .7  will  occur  with 
probability  .50582374 


Program  1. 

i 

This  program  tests  for  compliance  with 
a  minimum  reliability  standard. 

For  a  sample  size  of  70  items,  among 
which  exactly  20  defectives  were  observed, 
a  reliability  exceeding  .7  will  occur  with 
probability  .5754343 


Program  1. 

This  program  tests  for  compliance  with 
a  minimum  reliability  standard. 

For  a  sample  size  of  700  items,  among 
which  exactly  200  defectives  were  observed, 
a  reliability  exceeding  .7  will  occur  with 
probability  .7«98i566 
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This  program  computes  a  minimum  reliability 
consistent  with  a  specified  level  of  confidence. 

For  a  sample  size  of  7  items,  among 
which  exactly  2  defectives  were  observed, 
a  reliability  exceeding  .53790159  will 
occur  with  probability  .8 


Program  2. 

This  program  computes  a  minimum  reliability 
consistent  with  a  specified  level  of  confidence. 

For  a  sample  size  of  14  items,  among 
which  exactly  4  defectives  were  observed, 
a  reliability  exceeding  .59237177  will 
occur  with  probability  .8 


Program  2. 

This  program  computes  a  minimum  reliability 
consistent  with  a  specified  level  of  confidence. 

For  a  sample  size  of  70  items,  among 
which  exactly  20  defectives  were  observed, 
a  reliability  exceeding  .66385031  will 
occur  with  probability  .8 


Program  2. 

This  program  computes  a  minimum  reliability 
consistent  with  a  specified  level  of  confidence. 

For  a  sample  size  of  700  items,  among 
which  exactly  200  defectives  were  observed, 
e  reliability  exceeding  .69937687  will 
occur  with  probability  .8 


*  **  *.  *  •»  **  «•'  A  •  * 


Program  3. 


This  program  computes  a  "Best  Estimate" 
of  the  reliability  in  the  sense  of  minimum 
distance  between  bounds  for  a  specified 
level  of  confidence. 

For  a  sample  size  of  7  items,  among 
which  exactly  2  defectives  were  observed, 
the  "Best  Estimate"  of  the  reliability 
falls  between  .49573206  and  .87983592 
with  probability  .8 


Program  3. 

This  program  computes  a  "Best  Estimate" 
of  the  reliability  in  the  sense  of  minimum 
distance  between  bounds  for  a  specified 
level  of  confidence. 

For  a  sample  size  of  14  items,  among 
which  exactly  4  defectives  were  observed, 
the  "Best  Estimate"  of  the  reliability 
falls  between  .55515217  and  .8438256 
with  probability  .8 


Program  3. 

This  program  computes  a  "Best  Estimate" 
of  the  reliability  in  the  sense  of  minimum 
distance  between  bounds  for  a  specified 
level  of  confidence. 

For  a  sample  size  of  70  items,  among 
which  exactly  20  defectives  were  observed, 
the  "Best  Estimate"  of  the  reliability 
falls  between  .6428334  and  .77920786 
with  probability  .8 
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Program  4. 


This  prooran  tests  for  ccirrlience  with  a 
specified:  rraxJmum  number  cf  allowable  failures 
in  the  original  population. 

The  population  consists  cf  26  similar  items,  A 
sample  cf  size  7  ic  drawn  which  contains  exactly  2 
defectives.  The  original  population  contained  iewer 
then  $  defectives  with  probability  .EEC25C2? 


Prcgran  4  , 

This  pregrar  tests  fer  cciiplJfr.ce.  with  t 
spec  J  lice'  na&:nun  nun  ter  cf  allowable  failures 
Jr  the  criminal  population. 

The  pcFtlcticn  corslets  cf  26  einilar  iters#  A 
can  pic  cf  t  i  i  c  7  la  ersvr,  which  contains  exactly  2 
defectives,  The  criglral  pcpclrticn  cents] red  fewer 
than  12  defectives  with  probability  .lllf?*!8? 


Prcgrar  4. 

ThJr  jrregran  torts  for  ccppliar.ee  with  c 
specified  r,  P. x  ;r  ur  rurher  rf  allowable  fa  Hi  res 
in  the  original  pc  pul  a t icn  . 

The  population  ccrcJf.tr  cf  4F  rlrilar  iters,  P 
ran  pic  cf  rise  7  Jr  drawn  which  ccn tains  exactly  2 
defectives.  The  crigJral  population  ccntrJred  fever 
thar  16  defect iver  with  probability  .5229367 


Prccrem  4, 

This  program  tents  fer  ccitpllance  with  t 
specified  iraxin-urr  runher  cf  allowable  failure? 
in  the  original  peculation. 

The  population  ccnsiBts  cf  4F  similar  Items,  A 
sample  of  Eise  7  is  drawn  which  contains  exactly  2 
defectives.  The  original  population  contained  fewer 
than  22  defectives  with  probability  .8001343 


Program  5 


This  program  has  an  expanded  iprintcut  which 
tabulates  both  specific  and  cumulative  probabilities 
for  various  numbers  cf  failures.  The  upper  bound  of 
the  listing  .can  be  established  either  by  stating  the 
maximum  number  cf  allowable  failures,  cr  by  specify¬ 
ing  a  desired  cumulative  probability. 

The  population  consists  cf  26  similar  items* 
sanplc  of  size  7  it  drawn  which  contains  exactly  2 
defectives.  The  various  prcfceb i 1 i t i es  ere* 


Number  cf 

Probab  i  1  i  ty 

Cumulative 

fa i i uree 

c.f  cccurronce 

prebabil i ty 

2 

0.01214530 

C. 01914530 

t 

to 

0.0454700? 

0,06461538 

A 

0. 07117057 

0. 1 3578 55 5 

5 

0. 0fT  65907 

0.22744502 

6 

0.10475222 

0.33212824 

7 

0. 1C99PCG8 

0.44218912 

f 

C. 1C 5 061 21 

C. 55025033 

c 

to 

0.10034256 

C. 65059289 

1  c 

0.00053755 

0.73213043 

n 

C. 07439614 

•  0.  81.3  526  57 

12 

C . 05?  51691 

0.87304346 

] : 

0.04521735 

C. S182C007 

14 

0. 0324C377 

0. 95072  464 

15 

0.02105061 

0.27257525 

16 

0.01362116 

C. 98619641 

17 

0 . 00771066 

C. 99391507 

IP 

0.00385933 

0. 95777440 

1? 

0.00161751 

0 . 99939.1  91 

20 

0.00051350 

0.  «  f 920541 

;i 

0.00005459 

3 . 00000000 
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Prcgran  5, 

This  prcsran.  has  an  expended  printout  which 
tabulates  both  specific  arc  cumulative  probabilities, 
for  various  numbers  of  failures  •  The  upper  bound  cf 
the  listing  car  be  established  either  by  stating  the 
maximum  nuirber  cf  allowable  failures,  cr  by  specify¬ 
ing  a  desired  cumulative  probability. 

The  population  ccrrists  cf  4G  einilar  iterrs,  A 
sarrle  cf  eise  7  is  drawn  which  certains  exactly  2 
defectives,.  The  various  prcbatJ 1  it Jer  erc» 


Number  cf 

Prcbahl 1  i  ty 

C  uir  u  let  i  v  e 

fa i lures 

cl  occurrence 

prot  ah  j 1  i  ty 

2 

0. 00303951 

C. 00303951 

m 0 

G.00P12740 

0.01116691 

4 

0.C1444C7C 

0  .  02561  561 

c 

0.02134457 

0. 04*96029 

<! 

0.02C29410 

C. 07525439 

m 

1 

0 . 034696CC 

0.11015045, 

P 

0.040P5392 

0.15100436 

r 

0.04596057 

0 . 196P6504 

1  C; 

0.05008534 

0.24705030 

11 

0.05316076 

C. 30021114 

1 : 

0.05517224 

‘  0.35530330 

13 

C. 05614751 

0.41153089 

14 

0.05614751 

C. 46767840 

]  5 

0 .05525830 

0. 52293670 

16 

0.05358360 

0. 57652C50 

17 

0.05123951 

0.62776002 

16 

0.04R34696 

0.07610698 

19 

0.04502903 

0.72113601 

20 

0.04140600 

0.76254201 

21 

0.03759229 

0.80013430 

22 

0.03369383 

0.83382614 

23 

0.02980608 

0. 86363422 

24 

0 .02601258 

0.86964680 

25 

0.02238402 

0.91203082 

26 

0 .01897775 

0 .931008-57 

Program  6  . 


This  program  computes  the  maximum  number  of 
defectives  in  the  original  population  consistent 
with  a  specified  confidence  level  and  with  the 
observed  number  cf  sample  failures. 

The  population  consists  of  26  similar  items.  A 
sample  of  size  7  is  drawn  which  contains  exactly  2 
defectives.  The  original  population  contained  fewer 
than  12  defectives  with  probability  .81352657 
Or  fewer  than  11  with  probability  .73913043 


Program  6. 

This  program  computes  the  maximum  number  cf 
defectives  in  the  original  population  consistent 
with  a  specified  confidence  level  and  with  the 
observed  number  of  sample  failures. 

The  population  consists  of  48  similar  items.  A 
sample  of  size  7  is  drawn  which  contains  exactly  2 
defectives.  The  original  population  contained  fewer 
than  22  defectives  with  probability  .8001343 
Or  fewer  than  21  with  probability  .76254201 


Program  6, 

'  This  program  computes  the  maximum  number  of 
defectives  in  the  original  population  consistent 
with  a  specified  confidence  level  and  with  the 
observed  number  of  sample  failures. 

The  population  consists  of  125  similar  items.  I 
sample  of  size  14  is  drawn  which  contains  exactly  4 
defectives.  The  original  population  contained  fewer 
than  51  defectives  with  probability  .8094822 
Or  fewer  than  50  with  probability  .79092954 


I 


IS 
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Program  7. 


.This  program  yields  e  "Best  Estimate"  cf 
the  total  number  of  defectives  in  the  original 
population,  ir.  the  sense  cf  minimum  distance 
between  bounds  fcr  a  specified  ccnfidence  level. 

The  population  consists  of  26  similar  items, 

A  sample  of  size  7  is  drawn  which  contains  exactly 
2  defectives.  At  the  .80642809  level  of  confi¬ 
dence,  the  original  population  contained  between 
4  and  12  defectives,  inclusive. 


Program'  7. 

This  pregram  yields  a  "Eest  Estimate"  cf 
the  total  number  of  defectives  in  the  original 
population,  in  the  sense  of  minimus  distance 
between  bounds  fcr  a  specified  ccnfidence  level. 

The  population  consists  of  48  similar  Items. 

A  sample  cf  size  7  is  drawn  which  contains  exactly 
2  defectives.  At  the  .01667294  level  of  confi¬ 
dence,  the  orJeirel  population  contained  between 
6  and  23  defectives,  inclusive. 


Program  7. 

This  program  yields  a  "tscst  Estimate"  of 
the  total  number  cf  defectives  in  the  original 
pcpulat.  ion,  in  the  tense  cf  mir.  imun  distance 
between  bounds  for  a  specified  confidence  level. 

The  population  ccnsists  of  125  similar  items. 

A  sample  of  sire  14  is  drawn  which  contains  exactly 
4  defectives.  At  the  .80976263  level  cf  confi¬ 
dence,  the  original  population  contained  between 
20  anci  54  defectives,  inclusive. 


ThJe  program  has  an  expanded  printout  which 
tabulates  both  specific  and  cumulative  probabilities 
for  various  numbers  of  failures.  The  upper  and  lower 
bounds  of  the  listing  reflect  the  maximum  and  minimum 
\  number  of  failures  associated  with  the  "Best  Estimate'1 

j  consistent  with  a  stated  level  of  confidence. 

|  The  population  consists  of  2 6  similar  items. 

A  sample  of  size  7  is  drawn  which  contains  exactly 
'  2  defectives.  The  .various  probabilities  arei 


Number  of 

Probability 

Cumulative 

failures 

of  occurrence 

probabi  3.  J  ty 

4 

0.07117057 

0. 13578595 

5 

0.09165907 

0.22744502 

6 

0.10475322 

0. 33219824 

7 

0.10999086 

0.44218912 

8 

0.10606121 

0. 55025033 

9 

0.10034256 

0.65059289 

10 

0.08653755 

0.73913043 

11 

0.07439614 

,  0.81352657 

12 

0.05951691 

0.87304348 

Sum 


0.80842609 


IConf.  level) 


Program  8. 

This  program  has  an  expended  printout  which 
tabulates  both  specific  and  .cumulative  probabilities 
for  various  numbers  of  failures.  The  upper  and  lower 
bounds  of  the  listing  reflect  the  maximum  and  minimum 
number  of  failures  associated  with  the  "Best  Estimate' 
consistent  with  a  stated  level  of  confidence. 

The  population  consists  of  48  similar  items. 

A  sample  of  size  7  is  drawn  which  contains  exactly 
2  defectives.  The  various  probab i 1 i t ies  arei 


Number  of 
■  failures 


Probab i 1 ity 
of  occurrence 

0.02829410 

0.03489608 

0.04085392 

0.04596067 

0.05008534 

0.05316076 

0.05517224 

0.05614751 

0.05814751 

0.05525830 

0.05358380 

0.05123951 

0.04834696 

0.04502903 

0.04140600 

0.03759229 

0.03369383 

0.02980608 

0.81667394 


Cumulative 
prcbabili ty 

0.07525439 

0.11015045 

0.15100438 

0.19696504 

0.24705038 

0.30021114 
0.35538338 
0.41153089 
0.46767840 
0. 52293670 

0.57652050 
0.62776002 
0.67610698 
0. 72113601 
0.76254201 

0.80013430 

0.83382814 

0.86363422 

(Ccnf.  level) 
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This  program  yields  a  "Pest  Estimate"  cf 
the  total  number  of  defectives  in  the  original 
population,  in  the  sense  of  minimum  distance 
between  bounds  for  a  specified  confidence  level. 
Also  given  Je  the  maximum  likelihood  estimate. 

The  population  consists  of  26  similar  items. 

A  sample  of  size  7  is  drawn  which  contains  exactly 
2  defectives.  At  the  .80942809  level  of  confi¬ 
dence,  the  original  population  contained  between 
4  and  12  defectives,  inclusive. 

The  maximum  likelihood  estimate  is  7  defectives. 


Program  9 . 

This  program  yields  a  "Beet  Estimate"  cf 
the  total  number  of  defectives  in  the  original 
population i n  the  sense  of  minimum  distance 
between  bounds  for  a  specified  confidence  level. 
Also  given  Je  the  maximum  likelihood  estimate. 

The  population  consists  of  46  similar  items. 
a  sample  .cf  size  7  is  drawn  which  contains  exactly 
2  defectives.  At  the  .81667394  level  of  confi¬ 
dence,  the  original  population  contained  between 
6  and  23  defectives,  inclusive.  , 

There  are  two  equally  likely  maximum  likelihood 
estimates.  They  are  13  and  14  defectives. 


Program  9. 

This  program  yields  a  "Best  Estimate”  of 
the  total  number  of  defectives  in  the  original 
population,  in  the  sense  of  minimum  distance 
between  bounds  for  a  specified  confidence  level. 
Alao  given  ia  the  maximum  likelihood  estimate. 

The  population  consists  of  125  similar  items. 

A  sample  of  size  14  is  drawn  which  contains  exactly 
4  defectives.  At  the  .80976263  level  of  confi¬ 
dence,  the  original  population  contained  between 
20  and  54  defectives,  inclusive. 

There  are  two  equally  likely  maximum  likelihood 
estimates.  They  are  35  and  36  defectives. 
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Program  10 


This  program  tests  for  compliance  with  a 
minimum  mean  life  standard. 

For  a  test  duration  of  3200  hours 
during  which  exactly  6  failures  were  observed, 
a  mean  life  exceeding  500  hours 

will  occur  with  probability  .45767113 


Program  10. 

This  program  tests  for  compliance  with  a 
minimum  mean  life  standard. 

For  a  test  .durat ion  of  3200  hours 
during  which  exactly  5  failures  were  observed, 
a  mean  life  exceeding  500  hours 

will  occur  with  probability  .616256i34 


Pregram  10. 

This  program  tests  for  compliance  with  a 
minimum  mean  life  standard. 

For  a  test  .duration  of  3200  hours 
during  which  exactly  4  failures  were  observed, 
a  mean  life  exceeding  500  hours 

will  occur  with  probability  .76492997 


Program  10. 

This  program  teats  for  compliance  with  a 
minimum  mean  life  standard. 

For  a  test  duration  of  3200  hours 
during  which  exactly  3  failures  were  observed, 
a  mean  life  exceeding  500  hours 

will  occur  with  probability  .88108124 


■Mv.v?. 
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Program  11 . 

This  program  .computes  a  minimum  mean  life 
.consistent  with  a  specified  level  of  confidence. 

For  a  test  duration  of  .  3200  hours 
during  which  exactly  6  failures  were  observed, 
a  mean  life  exceeding  352.60211  hours 
will  occur  with  probability  .8 


Program  11. 

This  program  computes  a  minimum  mean  life 
.consistent  with  a  specified  level  of  confidence. 

For  a  test  duration  of  3200  hours 
during  which  exactly  5  failures  were  observed, 
a  mean  life  exceeding  404.75623  hours 
will  occur  with  probability  .6 


Program  11. 

This  program  computes  a  minimum  mean  life 
consistent  with  a  specified  level  of  confidence. 

i 

For  a  test  duration  of  3200  hours 
during  which  exactly  4  failures  were  observed, 
a  mean  life  exceeding  476.12113  hours 
will  toocur  with  probability  .8 


Program  11. 

This  program  computes  a  minimum  mean  life 
.consistent  with  a  specified  level  of  confidence. 

For  a  test  duration  of  3200  hours 
during  which  exactly  3  failures  were  observed, 
a  mean  life  exceeding  580.23091  hours 
will  occur  with  probability  .8 


LV. 
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Program  12. 


This  program  .computes  the  "Best  Estimated  of 
the  Mean  Life  .consistent  with  a  specified  Level 
of  Confidence. 

For  a  test  duration  of  3200  hours 
exactly  6  failures  were  observed. 

At  the  .8  Level  of  Confidence,  the 
"Best  Estimate"  of  the  Mean  Life  falls 
between  327.58165  and 
954.25284  hours.  The  maximum  like¬ 
lihood  estimate  is  533.33333  hours. 


Program  12. 

.This  program  .computes  the  "Best  Estimate^"  of 
the  Mean  Life  consistent  with  a  specified  Level 
of  Confidence. 

For  a  teat  duration  of  3200  hours 
exactly  5  failures  were  observed. 

At  the  .8  Level  of  Confidence,  the 
"Best  Estimate"  of  the  Mean  Life  falls 
between  376.27302  and 
1220.02901  hours.  The  maximum  like¬ 
lihood  estimate  is  640  hours. 


Program  12. 

This  program  .computes  the  "Best  Estimate"  .of 
the  Mean  Life  .consistent  with  a  specified  Level 
of  Confidence. 

For  a  test  duration  of  3200  hours 
exactly  4  failures  were  observed. 

At  the  .8  Level  of  Confidence,  the 
"Best  Estimate"  of  the  Mean  Life  falls 
between  443.51195  and 
1666.43711  hours.  The  maximum  like¬ 
lihood  estimate  is  800  hours. 
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APPENDIX  C 


Same  Notes  bn  BASIC  Programming 


Like  much  that  la  utilitarian,  tho  BASIC  program¬ 
ming  language*  la  In  a  state  of  constant  evolution. 
Moreovery  most  vendors  embellish  It  with  additional 
statements  and  commands,  Intended  to  solve  real  or 
Imagined  difficulties.  Borne  of  these  differences  are 
pointed  out  here.  But  the  ultimate  guide  is  a  good 
owner's  manual. 

1.  Capital  Letters.  Older  versions  of  BASIC  re¬ 
quire  the  use  of  capital  letters*  Some  later  ones  can 
recognice  instructions  written  in  lower  Case. 

2.  Vf.r.Uh.U.-fllllB.n.ffll.ni.*,.  O versions  require 
that  the  statement  LET  precede  the  variable  name. 

With  later  versions,  its  use  is  optional.  Variables 
should  be  assigned  one  at  a  time.  Xn  a  statement  such 
as  "A  ■  B  ■  4",  many  versions  would  regard  the  ascend 
equals  sign  as  a  logical  operator. 

3.  Variable  Names.  Most  versions  allow  names  of 
some  length  so  that  they  can  act  as  prompts  to  the  opsr1 
ator.  However,  as  few  as  two  characters  may  be  used  by 
the  machine.  Thus  "DIVER"  could  not  be  distinguished 
from  "DISCO".  Certain  reserved  words  --  such  as  FN« 

00,  IF,  ON,  OR,  TO,  and  others  --  should  be  avoided  «■ 
variable  names. 

4.  Subroutine  Calls.  Borne  versions  allow  palling 
a  subroutine  by  label.  Others  call  by  line  number  only 

5.  Multiple  statements  on  a  line,  If  allowad,  ere 
separated  by  sobs  symbol  whieh  varies  among  versions. 
Perhaps  i  \  and  i  are  the  most  common. 


4.  Remarks  are  variously  preceded  by  *  !  or 

REM. 

7.  Thoro  are  many  varlatiaa  of  truncation  at  tha 
radlN.  Ona  will  aneountar  XNT,  FIX*  IP,  PP,  FRC  and 
othara. 

0.  Parhapa  tha  most  confusing  ia  tha  command 
"PRINT".  In  soma  varaions  it  la  uaad  to  addraaa  tha 
printar,  in  othara  to  addraaa  tha  display.  To  addraaa 
tha  printar,  we  may  find  PRINT  PRINT#  LPRXNT  or 
PR#1.  To  addraaa  tha  display,  wa  might  aneountar  7 
DISP  PRINT  or  PR#0.  Additionally,  PRINT#  aoma- 
timaa  ia  uaad  to  addraaa  a  maaa  atoraga  davlca. 

9.  For  formattad  print-out,  aoma  varaiona  rafar- 

anea  a  numbarad  IMAOK  lino.  Othara  rafaranca  a  string 
variable.  Thus  Program  S,  linaa  40  and  42  might  raadi 
60  Y4  -  »  ###  #.########  #.########" 

42  LPRXNT  U0XN0  Y4|  K+A,  0(A),  C(A> 

10.  Note  that  punctuation  ia  not  nacaaaarily  tha 
aama  from  version  to  varalon. 
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THE  APPLICATION  OF  EXPERIMENTAL  DESIGN  TO 
EVALUATION  OF  MULTI -ECHELON  STOCKAGE  MODELS 


Carl  B.  Bates 

US  Army  Concepts  Analysis  Agency 
Bethesda,  Maryland 


ABSTRACT.  A  long-range  objective  of  the  Army  Is  to  develop  a 
comprehensive  stockage  methodology  which  Integrates  the  echelons  and 
optimizes  cost,  weapon  system  availability,  and  transportability.  As 
research  Into  multi-echelon  Inventory  models  progresses,  various  stockage 
policies  (models)  have  been  proposed  as  ways  to  best  satisfy  requirements 
and  constraints.  The  Concepts  Analysis  Agency  conducted  a  study  to 
evaluate  the  utility  and  effectiveness  of  two  proposed  multi-echelon 
stockage  policies.  A  part  of  the  study  Involved  a  sensitivity  analysis  on 
each  of  the  simulation  models  and  a  comparative  analysis  on  the  two  models. 
This  paper  discusses  the  experimental  design  employed  and  the  statistical 
analysis  results  of  the  sensitivity  analyses  and  the  comparative  analysis. 


1.  BACKGROUND.  Current  Army  supply  policies  do  not  relate  operations 
and  maintenance,  Army  (OMA)  funds  to  weapon  system  availability.  The 
long-range  objective  of  the  Office  of  the  Deputy  Chief  of  Staff  for 
Logistics  (ODCSLOG)  Is  to  develop  a  comprehensive  stockage  policy  which 
Integrates  all  echelons  of  support  activities.  The  Retail  Inventory 
Management  Stockage  Policy  (RIMSTOP)  Model  Is  the  current  DOD  Inventory 
stockage  policy.  Methodology  development  of  multi-echelon  Inventory  theory 
Is  ongoing.  The  Inventory  Research  Office  (IRO)  recently  developed  the 
Major  Assemblies  Stockage  System  (MASS)  Model.  MASS  Is  a  multi-echelon 
Inventory  stockage  model  which  uses  an  optimization  process.  It  Is  a 
derivative  of  the  Multi-Echelon  Technique  for  Recoverable  Item  Control 
(METRIC)  Model  developed  by  The  Rand  Corporation.  Before  further 
development  or  Implementation  of  MASS,  ODCSLOG  wanted  an  Independent 
assessment  of  MASS.  Ultimately,  the  US  Army  Concepts  Analysis  Agency  (CAA) 
was  requested  to  evaluate  MASS  and  compare  It  with  RIMSTOP.  The  Multi- 
Echelon  Stockage  Analysis  (MESA)  Is  documented  In  a  CAA  Study  Report* 
published  In  1984.  A  part  of  the  study  objectives  was  to  conduct 
sensitivity  analyses  of  MASS  versus  RIMSTOP.  This  paper  discusses  the 
sensitivity  analyses  performed  on  the  two  models. 


♦Blake,  Robert  T.,  et  al.,  Multl-Echalon  Stockage  Analysis  (MESA). 
CAA-SR-84-18,  Bethesda,  MD,  May  1984, 
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2.  EXPERIMENTAL  DESIGN  AMD  MODEL.  Because  both  models  were 
relatively  t ast  running,  250  to  300  runs  were  allowed  for  the  sensitivity 
analyses.  Ultimately,  eight  Input  factors  common  to  both  models  were 
selected  for  the  Investigation.  Three  levels  (low,  nominal,  and  high)  were 
selected  for  each  of  the  eight  Input  factors  shown  In  Table  1. 


Table  1.  Input  Factors 


Direct  Support  (DS)  Units 

A  -  Repair  cycle  time  (days) 

B  -  Demand  (failed  major  assemblies  removed) 
C  -  Percent  of  failed  assemblies  repaired 
locally 

D  -  Order  ship  time  (days) 


General  Support  (GS)  Unit 

E  -  Repair  cycle  time  (days) 

F  -  Demand  (failed  major  assemblies  removed) 
G  -  Percent  of  failed  assemblies  repaired 
locally 

H  -  Order  ship  time  (days) 


This  gave  38  ■  6,561  factor  level  combinations.  A  1/27  x  38  fractional 
factorial  design  was  developed  which  contained  243  design  points.  The 
design  permitted  testing  of  the  8  main  effects  and  the  28  first  order 
Interaction  effects.  The  fixed  effects  analysis  of  variance  (ANOVA)  model 
for  the  fractional  factorial  design  was; 


y*  P  +  A  +  B  +  C+  ...  +  H  +  AB+AC  +  AD+...  GH  +  HOI • 


The  dependent  variable  (y)  represents  the  model  output  variable  Inventory 
Cost;  y  Is  a  true  but  unknown  common  effect;  the  letters  A,  B,  ....  H 
represent  the  eight  model  input  factors;  HOI  represents  the  Higher  Order 
Interactions.  The  ANOVA  table  Is  Illustrated  In  Table  2.  The  design  may 
appear  somewhat  wasteful  because  only  128  of  the  total  242  degrees  of 
freedom  were  fitted.  The  design  was  employed,  however,  because  of  the 
desire  to  measure  all  first  order  Interactions. 


Table  2.  ANQVA  Table  for  1/27  x  38  Fractional 
Factorial  Design 


Source 

Degrees  of 
freedom 

Sum  of 
squares 

Mean 

squares 

F-ratlo 

A 

2 

SS  (A) 

MS  (A) 

MS (A) /MS (HOI) 

B 

2 

SS  B 

MS  B) 

MS  B) /MS  HOI 

C 

2 

SS  C) 

MS(C) 

MS(C)/MS(H0I) 

• 

H 

AB 

AC 

AD 


2 

4 

4 

4 


SS  H) 
SS  AB) 
SS  AC 
SS  AD 


MS  H) 
MS  AB 
MS  AC 
MS  AD 


MS(H)/MS(H0I) 
MS  AB)/MS(HOI) 
MS  AC  /MS  HOI 
MS ( AD ) /MS ( HO I | 


GH  4 

114 

Total  242 


Higher  order 
Interactions 


SS(GH)  MS(GH)  MS ( 6H ) /MS (HOI) 

SS(HOI)  MS (HOI) 

SS  (T) 


3.  ANALYSIS.  The  single  output  variable  of  Interest  was  the  cost  of 
Inventory  required  to  achieve  an  anticipated  level  of  operational 
availability.  Analyses  were  performed  for  two  end  Items  (the  M60A3  tank 
and  the  M561  truck).  The  data  were  obtained  from  1982  maintenance  records 
from  the  Vth  and  the  Vllth  Corp  In  USAREUR .  The  ANOVA  Illustrated  In  Table 
above  was  performed  on  the  M561  and  the  M60A3  data  from  both  models. 

Main  effects  and  two-factor  Interaction  effects  were  tabulated,  and 
significant  two-factor  Interaction  effects  were  graphically  Illustrated. 

The  significant  two-factor  Interactions  for  the  M561  system  and  the  M60A3 
system  are  shown  In  Tables  3  and  4,  respectively.  Table  3  shows  11 
Interactions  significant  from  MASS  and  9  Interactions  significant  for 
RIMSTOP.  All  main  effects  except  A  and  F  are  contained  in  the  Interactions 
shown.  The  main  effect  A  was  significant  for  MASS,  but  It  was  not 
significant  for  RIMSTOP.  Main  effect  F  was  not  significant  for  either 
models;  therefore,  based  upon  the  M561  data.  Input  factor  F  (GS  Demand) 
does  not  have  a  significant  Influence  upon  Inventory  cost  from  either 
model. 


Table  3.  Significant  Interaction  Effects  for  M561 


BC  -  DS  assembly  removals  x  DS  percentage  repaired  0.001  0.001 

BD  -  DS  assembly  removals  x  DS  order  ship  time  0.001  0.001 

BE  -  DS  assembly  removals  x  GS  repair  cycle  time  0.05 
BG  -  DS  assembly  removals  x  GS  percentage  repaired  0.001  0.001 

BH  -  DS  assembly  removals  x  GS  order  ship  time  0.001  0.001 

CD  -  DS  percentage  repaired  x  DS  order  ship  time  0.001  0.05 

CG  -  DS  percentage  repaired  x  GS  percentage  repaired  0.01  0.05 

CH  -  DS  percentage  repaired  x  GS  order  ship  time  0.001  0.001 

DE  -  OS  order  ship  time  x  GS  repair  cycle  time  0.05  0.05 

EG  -  GS  repair  cycle  time  x  GS  percentage  repaired  0.001 

GH  -  GS  percentage  repaired  x  GS  order  ship  time  0.001  0.001 


Table  4.  Significant  Interaction  Effects  for  M60A3 


BC  -  DS  assembly  removals  x  DS  percentage  repaired  0.001  0.001 

BD  -  DS  assembly  removals  x  DS  order  ship  time  0.001  0.001 

BE  -  DS  assembly  removals  x  GS  repair  cycle  time  0.01  0.01 

BG  -  DS  assembly  removals  x  GS  percentage  repaired  0.001  0.001 

BH  -  DS  assembly  removals  x  GS  order  ship  time  0.001  0.001 

CD  -  DS  percentage  repaired  x  DS  order  ship  time  0.001 

CE  -  DS  percentage  repaired  x  GS  repair  cycle  time  0.05 

CG  -  DS  percentage  repaired  x  GS  percentage  repaired  0.001  0.001 

CH  -  DS  percentage  repaired  x  GS  order  ship  time  0.001  0.001 

DE  -  DS  order  ship  time  x  GS  repair  cycle  time  0.01  0.06 

EG  -  GS  repair  cycle  time  x  GS  percentage  repaired  0.001  0.001 

GH  -  GS  percentage  repaired  x  GS  order  ship  time  0.001  0.001 


Table  4  for  the  M60A3  data  shows  there  were  12  significant 
Interactions  for  MASS  and  10  for  RIMSTOP.  Most  of  the  Interactions  In 
Table  4  are  the  same  as  those  In  Table  3;  however,  the  two  lists  are  not 
Identical.  Again,  all  Input  factors  are  contained  In  the  Interactions  in 
Table  4  except  A  and  F.  This  time  the  F  main  effect  Is  not  significant  for 
MASS,  but  It  Is  for  RIMSTOP.  As  with  the  M561  data,  the  three  largest 
Interactions  In  Table  4  are  GH,  BH,  and  BC.' 

Figures  1  and  2  Illustrate  the  GH  (GS  percentage  repaired  x  GS  order 
ship  time)  Interaction  for  the  MS61  data  for  MASS  and  RIMSTOP, 
respectively.  The  figures  show  the  similarity  of  the  Interactions  from  the 
two  models.  Figures  3  and  4  show  the  same  Interaction  (GH)  for  the  M60A3 
and  the  two  models.  Again,  similarity  of  the  Interactions  Is  shown. 

Figures  1  and  3  show  the  comparison  of  the  M561  with  the  M60A3  data  for  the 
MASS  Model.  Figures  2  and  4  show  the  comparison  of  the  two  data  sets  for 
the  RIMSTOP  Modal . 

4.  SUMMARY.  The  utilization  of  experimental  design  permitted  an 
efficient  evaluation  of  the  MASS  Modal  and  a  comparison  of  MASS  with  the 
RIMSTOP  Model.  Experimental  design  should  be  an  Integral  part  of  the  test 
and  evaluation  of  computer  simulation  models.  Analysts  Involved  In  the 
exercise  of  simulation  models  for  the  purpose  of  generating  data  for 
analysis  should  be  apprised  of  the  need  for  experimental  design  and 
continually  reminded  of  Its  Importance. 
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Abstract i  This  "work  in  progress"  paper  describes  the  background _  and 
'development  of  a  distinctly  new  methodology  for  the  design  of  tests  and 
objective  evaluations  of  logistios  systems.  The  method  is  based  on  the 
physical  principle  known  as  Fermat1 s  Principle  of  Least  Time,  and  on  the 

praotioal  reality  that  "supplies  dolivered  too  late  are  worthless.  The 

problem  of  logistical  support  is  formulated  mathematically  using  variational 
calculus.  Rather  than  rigorously  solving  an  abstract,  general  problem,  an 

information  theory  approaoh  is  used  to  ohart  the  flow  of  logistioal  materials 
and  requests  in  time  and  space,  segment  the  flow  into  discrete  serial  units, 

and  analyse  eaoh  unit  separately.  In  this  way,  both  the  speoific  flow  unit 

where  a  delay  in  delivery  ooours  and  the  reason  for  the  delay  can  be 
identified.  The  optimal  solution  is  aohieved  whan  the  materials  pass  through 

all  serial  units  in  the  least  possible  time.  Tests  can  be  designed,  and 

oriteria  for  evaluation  of  the  tests  oan  be  constructed  by  comparing  measured 
results  to  the  optimal  solution.  In  particular,  the  method  oan  be  used  to 

produoe  speoifio  "target  values"  which  constitute  an  objective  yardstick  for 

evaluation. 


j.  Introduction.  The  design  and  oonduot  of  any  test  of  new,  improved,  or 
modified  Army  maUriel,  oonoepta,  foroe  struoture,  or  dootrine  ere  governed  by 
a  set  of  issues  and  oriteria  {Stevens,  1979).  Tto  issues  are  simply  questions 
about  the  performanoe,  reliability,  safety,  usability,  efficiency,  or  other 
aspects  of  the  tested  item.  The  oriteria  are  objeotive  standards  which  the 
tested  item  must  meet  if  it  is  to  be  Judged  acceptable  for  use  (and  for 
proourement/diatribution  subsequent  to  testing).  There  is  a  rule  or  thumb 
thst  says  that  the  beat  test  and  the  most  objeotive  evaluation  will  result 
when  the  criterion  is  one  which  oan  be  quantified  and  the  test  oan  produoe 
quantitative  results  for  comparison  to  the  oriterion.  Oiantitstive  oriteria 
are  often  referred  to  in  the  test/evaluation  literature  as  measures  of 
effectiveness  (HOE)  or  measures  of  performance  (MOP),  although  these  terms  are 
often  used  more  loosely  than  they  should  be  (l.e.,  to  denote  subjeotive  or 
non-quantitative  criteria).  Together  the  issues  and  oriteria  form  the 
building  blooks  whioh  govern  the  design  of  the  test,  execution  of  the  test 
plan,  and  analysis  of  the  test  data. 

Quantitative  oriteria  usually  are  obvious  or  follow  logioally  from  an 
examination  of  the  tested  item’s  purpose  or  intended  use.  For  instance,  iT  it 
is  desired  to  have  a  larger-oaliber  howitzer  because  more  range  is  needed,  one 
of  the  test  criterion  should  speak  to  the  amount  of  improvement  in  range 
(either  absolute,  in  km,  or  percent)  whioh  will  be  expeoted  when  the  new 
howitzer  is  oompared  to  the  one  it  will  replace.  Indeed,  for  most  functional 
areas  in  the  Army  (infantry  weapons,  tube  artillery,  ground  communications, 
etc.),  a  set  of  "generio"  issues  and  oriteria  oan  be  developed.  When  the  test 
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item  is  prepared,  the  specific  performance  numbers  can  be  added  to  the  generic 
criteria  and  the  testing  can  proceed.  Adherence  to  the  criteria  will  be 
especially  important  for  some  issues,  depending  on  the  function  of  the  tested 
item;  these  Issues  are  designated  as  neritioaln  issues  and  the  system  will  not 
be  judged  acceptable  unless  it  meets  all  the  criteria  for  all  critical  issues. 
Thus  the  importance  of  having  numerical,  objective  criteria  for  critical 
Issues  is  clear. 

In  some  Instances,  however,  it  is  not  at  all  easy  (even  with  logic  and 
common  sense!)  to  determine  objective  or  numerical  oriteria  for  a  given  test 
item.  This  is  particularly  true  for  items,  systems,  or  ooncepts  in  logistics 
and  supply.  Part  of  the  difficulty  arises  because  logistics  outs  aorosa  all 
functional  areas  rather  than  having  a  single  functional  area  of  its  own  with  a 
well-defined,  limited  mission.  The  question  of  logistics  supportabllity 
arises  in  the  consideration  of  all  Army  systems,  but  the  truth  of  the  matter 
Is  that  it  is  never  really  addressed  properly  in  testing.  Typically,  the  only 
logistics  issue  which  is  usually  written  is: 

Issue:  Is  the  test  system  loglstically  supportable? 

And  its  associated  criterion  is  almost  always: 

Criterion:  The  system  must  be  logistioajly  supportable. 

From  a  logical  point  of  view,  the  issue  and  associated  criterion  are 
unassailable;  the  Army  cannot  taotlcally  or  economically  afford  a  system  that 
the  logistics  supply  system  cannot  support,  no  matter  how  well  the  system 
works.  However,  it  is  equally  obvious  that  this  issue  and  orlterion  are  badly 
flawed  on  several  counts.  First,  the  criterion  begs  the  issue  and  does  not 
readily  admit  any  other  oonoelvable  answer,  which  violates  the  principle  of 
objectivity  in  test  design.  Secondly,  the  criterion  oannot  be  quantified,  and 
as  the  issue  is  written  no  quantifiable  orlterion  can  be  readily  determined. 
These  two  faotors  alone  would  preclude  any  reasonable,  objective,  believable 
test  of  the  system's  logistio  supportabllity.  TRADOC  Independent  Evaluation 
Directorate  (Ft.  Leavenworth)  has  attempted  on  at  least  two  occasions  to 
alleviate  this  problem  by  constructing  generlo  issues/criteria  for  logistic 
supportabllity,  with  appropriate  measures  of  performance  and/or  effectiveness, 
but  these  efforts  have  not  succeeded  because  o.  the  exceptionally  broad  soope 
of  logistios  problems  encountered  in  Army  test  and  evaluation. 

Hidden  within  the  problem  of  testing  logistic  supportabllity  is  a  dilemma 
which  is  neither  widely  recognized  nor  completely  understood  by  test 
designers.  It  is  true  that  a  system  should  not  (and  in  fact,  often  oannot)  be 
issued  to  the  Army  in  the  field  unless  logistio  supportabllity  is 
demonstrated.  Logistic  supportabllity  cannot  be  demonstrated  until  the  total 
complement  of  systems  is  fielded  and  the  total  supply  system  is  exercised. 
However,  for  testing  and  evaluation  only  a  few  systems  (typically  five  or  ten) 
are  procured,  so  the  logistics  support  systems  and  procedures  are  never  fully 
exercised.  This  creates  a  chicken-and-egg  dilemma:  the  total  number  of 
systems  desired  must  be  purchased  and  deployed  in  order  to  demonstrate 
logistio  supportabllity,  but  the  deoision  to  buy  more  than  just  a  few  systems 
for  testing  cannot  properly  be  made  until  logistic  supportabllity  is 
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demonstrated  by  a  test!  This  dilemma  would  be  solved  if  a  method  for 
assessing  logistic  supportability  using  a  limited  number  of  systems  could  be 
fo  und . 

The  answer  to  both  these  problems  —  non-quantifiable  oriterla  and  the 
dilerma  of  logistio  supportability  demonstration  —  appears  to  be  in  finding 
an  alternate  way  of  looking  at  the  loglstios  supply  problem.  "Conventional" 
logistics  analysis  more  or  less  relies  on  examining  the  quantity  of  supplies 
in  the  supply  system  and  their  availability  at  various  supply  points. 
Quantitative  analytical  tools  and  measures  of  effectiveness  exist  for 
determining,  how  well  the  system  works  (Engel,  1980).  However,  the  proper  use 
of  these  quantitative  measures  and  tools  depends  on  the  full  utilization  of 
the  logistics  supply  system,  which  again  implies  full  fielding  of  the 
materiel.  Thus  conventional  analytical  tools  are  not  expeoted  to  be  useful  in 
resolving  problems  which  fall  outside  of  the  situations  for  which  they  were 
designed,  and  the  implication  is  that  new  tools  will  have  to  be  developed. 

II,  Development  of  Least-Time  Analysis.  There  is  an  alternate  way  of  looking 
at  the  logistics  supply  problem.  Instead  of  a  conventional  or  "inventory" 
analysis,  in  which  the  functioning  of  the  supply  system  is  determined  by 
monitoring  the  supplies  on  hand  at  certain  points  in  the  system,  the 
alternative  is  to  examine  the  delivery  time  for  supplies  and  analyze  the 
effects  of  the  loglstios  supply  system  on  the  delivery  time.  On  their  faoe, 
these  two  analyses  would  appear  to  be  quite  different;  however,  with  a  little 
thought  one  can  convlnoe  oneself  that  in  fact  they  are  really  two  ways  of 
looking  at  the  same  thing  and  should  give  the  same  results.  The  advantage  to 
avoiding  any  "inventory"  analysis  is  that  without  the  constraints  of  counting 
supplies,  the  method  should  be  easily  extended  to  any  quantity  of  supplies, 
large  or  small,  and  the  supportability  dilemma  may  be  overoome. 

Using  an  analysis  based  on  delivery  time  has  the  distinot  advantage  that 

it  oan  embraoe  a  "guiding  principle"  of  logistics  supply  that  every' soldier  in 
the  field  knows:  Supplies  and  logistical  materials  delivered  too  slowly  are 
worth  no  more  than  no  supplies  at  all.  To  put  this  simply  but  graphically, 
the  ammunition  delivered  to  the  soldier  in  the  foxhole  is  of  no  use  if  the 

supply  system  is  so  slow  that  the  soldier  is  killed  before  he  gets  it.  This 

principle  provides  powerful  motivation  for  the  development  of  the  method  of 
Least-Time  Analysis. 

The  first  step  in  Least-Time  Analysis  is  illustrated  in  Figure  1.  Using  a 
map  model,  the  flow  of  supplies  from  manufacturer  (1)  to  depot  (2)  to  port  (3) 
to  foreign  port  (M)  to  staging  area  (5)  to  the  using  unit  (6)  is  plotted  and 
the  distances  and  travel  times  (inoluding  loading,  unloading,  refueling,  and 
all  delays)  ere  noted.  For  convenience,  the  map  model  (upper  half  of  Figure 
1)  is  converted  to  a  linear  delivery  model  (lower  half  of  Figure  1)  involving 
only  times  and  locations.  This  idea  is  borrowed  from  information  theory  and 
is  also  oalled  a  Shannon  model.  The  Shannon  model  is  appropriate  for  thin 
because  the  flow  of  supplies  can  be  modeling  as  linear  or  serial,  supplies 
flow  in  discrete  units  or  packages  which  can  be  traced,  and  the  locations  of 
the  supply  terminals  are  fixed  but  the  travel  time  between  them  is 
unconstrained . 


Once  the  times  Tl,  T2, ...  have  been  determined  from  the  linear  delivery 
model,  a  Space-Time  diagram  is  plotted  as  in  Figure  2.  The  location  of  eaoh 
"node"  in  the  linear  delivery  model  is  plotted  on  the  horizontal  (X)  axis  and 
is  treated  as  a  discrete  variable  with  only  certain  allowed  values  (the 
dlsoreta  values  oome  from  noting  that  depots,  ports,  etc.  are  fixed  in 
location  and  do  not  move  frequently).  Eaoh  location  is  represented  on  the 
graph  by  a  vertical  line.  Time  Is  treated  as  a  continuous,  cumulative 
variable  and  is  plotted  on  the  vertical  (Y)  axis.  As  shown,  the  time  it  takes 
for  the  supplies  to  pass  between  two  nodes  is  plotted  on  the  graph,  and  a 
curve  is  obtained  from  which  the  total  time  (called  delivery  time)  for 
supplies  to  travel  from  the  start  point  to  the  end  point  oan  be  determined. 
Delays  at  any  looatlon  are  represented  as  vortioal  gaps  whioh  make  the 
space-time  diagram  discontinuous. 

III.  Analysis  using  the  Space-Time  Diagram.  After  the  spaoe-time  diagram  is 
constructed  for  tEe  situation  being  considered,  it  is  analyzed  aooording  to 
three  rules.  These  rules  are  based  on  the  "guiding  principle"  of  logistios 
defined  two  paragraphs  previously. 

a)  The  optimal  path  for  moving  logistical  materiel  is  the  path  for  vMoh 
the  spaoe-time  diagram  gives  the  shortest  delivery  time. 

b)  Eaoh  segment  of  the  optimal  path  is  self-optimal (  that  is,  each  path 
segment  by  itself  takes  the  minimum  possible  time  to  go  from  its 
beginning  point  to  its  end  point. 

c)  Any  path  segment  whioh  does  not  take  the  minimum  time  is  not  optimal. 

It  follows  from  these  rules  that  test ,'  evaluation,  and  modification  activities 
need  to  be  directed  only  (and  especially)  toward  those  activities  whioh  make 
up  the  path  segments  whioh  are  identified  as  not  optimal.  These  segments  oan 
almost  always  be  identified  as  the  "choke  points"  whioh  hinder  the  proper  and 
timely  flow  of  logistical  materials. 

IV.  Mathematical  Foundation  for  Least-Time  Analysis.  Least-time  analysis  was 
inspired  by  a  well-known  principle  of  physios,  Fermat's  Principle  of  Least 
Time  (Marion,  1970).  Fermat's  principle  was  originally  applied  to  optical  ray 
traoing,  and  later  to  classical  kinematies  and  dynamics  of  moving  bodies.  It 
states  that  if  a  system  is  initially  in  one  state  or  condition,  and  at  some 
later  time  is  in  a  different  state  or  condition,  the  path  taken  to  go  from  the 
initial  state  to  the  final  state  will  be  the  path  which  allows  the  transition 
to  ooour  in  the  shortest  possible  time  period.  A  formalism  called  variational 
calculus  was  later  developed  to  desorlbe  these  kinds  of  transitions  between 
states.  Variational  oaloulus  is  extensive  and  complex j  only  a  few  details 
neoessary  to  form  the  basis  of  least-time  analysis  will  be  presented  here. 
The  notation  follows  the  basic  text  on  the  subject  (Gel fend  and  Fomin,  1963), 
Bnd  the  reader  is  referred  to  this  book  for  further  details. 


Consider  the  problem  Illustrated  here*  a  ball  is  to  roll  frictionlessl y 


on  some  path  between  points  A  and  B 


The  path  which  allows  the  ball  to 
oomplete  the  trip  in  the  least 
possible  time  must  be 
determined.  This  famous 

problem  (known  as  tho 
brachistochrone  problem)  Is 
solved  in  many  textbooks,  and 
is  referred  to  here  beoouse  It 
illustrates  the  basic 
structure  of  all  such 
problems,  Both  coordinates 
are  constrained;  the  looations 
of  points  A,  B  are  fixed  in 
.  both  x  and  y.  In  the '  y 
direotion,  the  additional 
constraint  of  a  fixed  force 
(gravity)  is  imposed.  These 
type  problems  are  solved  by 
defining  a  functional 

F(y(x),x;y'GO) 


where  y(x)  defines  tha  ourva  whioh  provides  tha  least  travel  time  between  A 
and  B  (called  the  optimal  path),  and  y?  is  defined  by 


y*(x)  •  y(x)  +  cm(x)  0 ) 


y(x)  la  assumed  to  be  continuous,  and  y’(x)  represents  tha  functions  whioh 
describe  all  paths  between  A  and  B  whioh  are  nett  tha  optimal  path  y( x) •  Thus, 
h(x)  represents  a  Mdiffarenoew  funotion  beVwaan  any  non-ideal  path  and  the 
ideal  path,  a'  is  assumed  to  be  small. 

Hie  task  appears  complicated  beoauae  wt  do  not  know  tha  optimal  path  y(x), 
but  rather  are  trying  to  aolva  for  it.  Reoognlslng  thatn(x)  oan  have  an 
infinite  number  of  forms,  tha  aquation  to  aolva  la 


J  -  f  P(y(x),xjy'(x))dx 

u 

(2) 

so  that 

a  j  1  _ 

■  o. 

(3) 

9a  a+o 

These  two  equations  basically  say  tha  following i  Examlna  all  possible 
paths  y’(x).  For  eaoh  one,  look  at  tha  first  derivative  of  J  over  the  entire 
path  and  find  the  extremal  path  (first  derivative  equal  to  zero).  If  the 


first  derivative  becomes  2ero  when  the  value  of  «  becomes  zero,  the  function 
y'(x)  is  actually  y(x),  the  optimal  path  we  are  searching  for. 

Gelfand  and  .Fomin  demonstrate  that  the  problem  la  mathematically  solvable 
for  virtually  all  yC  x)  and  n(x),  and  they  desoribe  exhaustively  how  it  is 
done.  For  the  purpose  of  least-time  analysis,  only  3  results  need  to  be 
noted  I 

a)  There  la  always  an  optimal  path,  and  it  oan  always  be  found. 

b)  The  optimal  path  is  unique)  there  oannot  be  more  than  one  ''optimal" 
path  for  a  given  set  of  constraints. 

a)  If  y(x)  is  not  oontlnuous  but  ia  piecewise  oontlnuous  with  n  pieces, 
the  optimal  path  is  found  by  substituting  for  Ji 

r°l  fB 

J  -  F(  )dx  +  F<  )dx  +  •••  +  F<  )dx  .  (H) 

U  01  n 

This  says  that  if  a  path  is  not  oontlnuous,  the  optimal  path  oan  still  be 
found  by  finding  the  optimal  path  for  each  segment  whioh  is  oontlnuous. 

I 

This  is  sufficient  to  establish  a  mathematical  basis  for  least-time 
analysis.  Although  the  methodology  dearly  belongs  to  the  olass  of  tools 
known  as  '"critical  path"  methods,  its  development  and  motivation  are  quite 
different  from  most  methods  in  this  elass.  In  faot,  least-time  analysis 
appears  to  be  the  first  orltioal  path  method  derived  solely  from, a  variational 
oaloulus  approach. 

Before  leaving  this  section,  it  is  appropriate  to  compare, least-time 
analysis  with  another  and  perhaps  more  familiar  orltioal  path  method,  Program 
Evaluation  Review  Technique  (PERT).  As  applied  to  loglstlos  and  similar 
scheduling  problems,  PERT  attempts  to  identify  the  slowest  among  multiple 
paths  so  that  coordination  can  be  effeoted  whioh  will  bring  all  the  palhsto  a 
common  end  point  at  the  same  time.  Least-time  analysis,  in  contrast,  takes  a 
situation  where  a  single  path  exists  and  attempts  to  Identify  the  slowest 
segment  (or,  perhaps ,"  tne”slowest  among  possible  alternate  paths)  in  order  to 
reduce  the  time  required  for  (and  thus  optimise)  the  total  path*  Beoause  of 
this  essential  difference,  least-time  analysis  appesrs  to  have  at  least  two 
advantages  over  PERTi  the  date  requirements  ars  less,  and  the  answer  derived 
from  least-time  analysis  is  unique  (there  Is  only  one  optimal  path), 

V.  Application  of  Lesst-Tlms  Analysis.  It  is  not  necessary  to  use  the 
extensive  mathematical  formalism  behind  least-time  analysis  in  order  to  obtain 
useful  results  (although  the  existence  of  a  mathematical  background  Is 
tantalizing  for  logistics  modelers).  Application  of  least-time  analysis 
requires  basloally  four  steps. 

First,  the  level  of  application  must  be  determined.  Depending  on  how  much 
detail  ia  desired  in  the  analysis,  the  method  can  be  applied  to  parts, 
subassemblies,  systems,  supply  classes,  or  adjuncts  to  systems. 
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Next,  a  linear  delivery  model  (Shannon  model;  see  figure  1)  is  constructed 
for  the  supply  path  at  the  chosen  level  of  application.  This  model  is  used  to 
identify  eaoh  handling  step  or  path  segment  in  the  supply  prooess. 

Next,  the  time  required  for  eaoh  step  is  determined.  This  determination 
oan  be  made  by  "desk-top"  analysis,  literature  research,  or 
testing/ observation.  When  these  times  are  known,  a  spaoe-time  diagram  auoh  as 
figure  2  oan  be  plotted. 

Finally,  from  the  spaoe-time  diagram  and  the  principle  of  analysis  given 
in  seotlon  III,  all  path  segments  whloh  are  not  believed  to  be  optimal  oan  be 
identified.  If  the  time  required  for  eaoh  step  wss  determined  by  a  literature 
searoh  or  by,  desk-top  analysis,  appropriate  issues/orlterla  for  testing  oan  be 
written.  If  the  times  required  for  eaoh  step  were  found  bytesting  or 
observation,  they  oan  be  analyzed  to  see  if  new  issues/oriteria  or  further 
testing  will  be  neoessary. 

In  addition  to  being  potentially  useful  for  attaoking  problems  whloh  are 
not  easily  handled  by  other  methods,  least-time  analysis  fits  nicely  within 
the  conventional  framework  of  loglstioa  and  thus  allows  quantification  of  some 
concepts  whloh  are  widely  considered  to  be  only  qualitative  in  nature.  For 
lnstanoe,  the  loglstlolan  speaks  of  the  three  products  of  logistlost 
readiness  in  time  of  peace,  surge  at  the  outbreak  of  hostilities,  and 

sustainability  until  the  fighting  is  ended.  Intuitively  the  meaning  of  these 
terms  is  obvious,  but  they  have  not  previously  been  quantified  for  use  by  the 
test/ evaluation  oommunlty.  Figure  3  illustrates  how  least-time  analysis 
oomblnes  with  a  little  oommon  sense  to  make  these  terms  quantitative. 
Readiness  represents  the  "relaxed"  or  "natural"  state  of  the  supply  system, 
with  loglstlos  materiel  being  delivered  only  at  the  rate  required,  for  normal 
operation  of  the  military  unit,  Henoe  the  required  times  for  eaob/btep  (and 
the  overall  delivery  time)  oan  be  determined  reliably  and  with  high  preodslon 
from  simple  analysis  of  reoords  already  existing.  In  the  surge  mode,  the 
ourve  on  the  spaoe-time  diagram  flattens  out  greatly  because  the  delivery  time 
for  any  materiel  must  deorease  (note  how  this  is  equivalent  to  the 

conventional  method  of  representing  surge  as  a  drastlo  lnoreaae  in  the 
quantity  of  all  supplies).  In  the  sustainability  mode,  the  ourve  is  not  as 
flat  beoause  the  delivery  of  supplies  is  not  so  orltioal  as  in  the  initial 
surge,  but  the  delivery  time  must  still  be  shorter  than  in  the  readiness 
phase.  Using  figure  3,  the  loglatlolan  oould  now  quantify  these  -  for 
lnstanoe,  by  requiring  the  delivery  time  during  surge  to  be  one-half  of  the 
delivery  time  during  readiness.  This  gives  the  tester  a  quantitative  way  to 
evaluate  the  loglstlos  supply  system. 

It  is  also  of  interest  to  note  that  some  other  types  of  information  may  be 
extraoted  from  a  least-time  analysis.  For  example,  some  estimate  of 
vulnerability  of  a  system  oan  be  obtained  by  reoognlzlng  that  the  steps  on  the 
supply  path  whloh  require  more  time  are  more  vulnerable  to  enemy  attaok  and 
disruption  than  steps  whloh  proceed  more  rapidly.  The  oaveat  to  this  is  that 
vulnerability  is  actually  more  oomplex  and  involves  more  faotors  than  simply 
the  time  spent  on  one  path  or  in  one  plaae,  so  any  vulnerability  derived  from 


least-time  analysis  is  striotly  an  estimate  and  should  be  used  only  with  great 
oare . 

VI,  An  Examplet  Logistics  over  the  Shore,  Logiatioa  over  the  shore  (LOTS) 
a  major  Army-Navy  joint  test  tdiloh  presented  partioular  problems  to  the 
test  community  because  it  initially  lacked  good  issues,  oriteria,  and  meaaures 
of  effectiveneas  or  performance.  Some  retrospective  snalysis  of  LOTS  using 
least-time  analysis  will  illustrate  the  usefulneaa  of  the  method.  LOTS 
represented  the  sustainment  phase  of  logiatios,  with  the  beaohhead  secured  and 
the  threat  fran  enemy  fire  or  troops  to  logistics  units  not  significant* 
Using  several  fully-loaded  ships,  the  exercises  made  use  of  all  possible 
methods  to  unload  oargo  of  all  types  and  move  it  to  the  beaches  in  the  moat 
expedient  way  possible.  The  test  was  to  answer  two  general  questional 

a)  What  is  the  most  effloient  method  among  all  possible  ways  to  transfer 
oargo  from  ship  to  shore? 

b)  Where  la  improvement  needed  in  all,  methods  to  make  them  more 
efficient? 

The  results  of  the  test  were  assigned  speolal  significance  because  they 
would  be  used  to  rewrite  dootrine  for  beaohhead  supply  and  standard  operating 
procedures  (SOPs)  for  logiatios  units  in  shlp-to-shore  operations,  make 
revisions  to  tables  of  organisational  and  equipment  (TOEs),  make  possible 
ohangea  in  force  structure,  and  support  production  decisions  for  amphibious 
lightera  and  other  shlp-to-shore  wateroraft.  Beoause  of  spaoe  limitations, 
the  example  considered  here  ia  somewhat  abbreviated  and  does  not  represent  the 
entire  LOTS  exerolaes. 

The  decision  was  made  to  break  down  the  analysis  to  the  "type  of  oargo" 
level,  and  aooordlngly  the  first  two  classes  to  be  considered  were  breakbulk 
oargo  and  containerised  (non-liquid)  oargo.  Figura  4  shows  the  linear 
delivery  model  for  breakbulk  oargo,  oonstruoted  to  show  all  possible 
combinations  of  handling  methods  used  to  transfer  the  oargo  from  ship  to 
shore.  Analysis  then  shows  that  containerised  oargo  is  handled  in  axaotly  the 
same  ways,  so  the  linear  delivery  model  would  be  the  same. 

For  breakbulk  oargo,  several  questions  oan  immediately  be  generated  from 
figure  4,  These  lnoludei 

a)  Which  path  represents  the  most  effloient  delivery  method? 

b)  What  improvements  oan  be  made  to  esoh  path  to  deorease  its  delivery 
time? 

o)  What  safety  hasards  are  present  on  eaoh  path? 

d)  Is  sufficient  equipment  and  rigging  available  to  exeoute  all  paths 
without  delay? 


And,  when  containerized  oargo  is  considered,  the  above  questions  remain 
and  there  are  additional  questions t 

a)  Is  it  more  efficient  to  move  containerized  oargo  than  breakbulk  oargo? 

f)  What  improvements  to  breakbulk  oargo  handling  would  inorease  its 
effioienoy  to  be  competitive  with  containerized  oargo  handling? 

A  combination  of  testing  and  desktop  analysis  would  be  used  to  construct 
spaoe-time  diagrams  for  each  path  and  for  eaoh  cargo  type,  end  from  these 
diagrams  the  questions  listed  above  oould  be  turned  into  testable  Issues  with 
quantitative  orlterle.  As  either  a  test  planning  aid  or  a  data  analysis  aid, 
least-time  analysis  is  very  useful  beoause  the  problem  oould  not  easily  be 
attaoked  with  any  other  method  available.  The  analysis  would  be  oontlnued  in 
the  same  manner  for  all  other  types  of  oargo  (gasoline,  vehiolts,  ate.). 

VII,  Conclusion,  As  stated  in  the  abstract,  leatt-time  analysis  is  still  in 
development,  it  has  demonstrated  Its  usefulntss  in  generating  Issues  and 
oriterla  for  logistics-related  testing,  and  it  shows  some  promise  of  being 
useful  to  logistics  systems  designers  and  modelers.  Muoh  of  its  utility  oomes 
from  the  property  of  fitting  into  a  oonventionsl  framework  of  logistics  (that 
is,  having  the  same  point  of  view  and  not  requiring  new  language  or  ways  of 
thinking  about  loglstloa)  but  taking  a  somewhat  different  approaoh  than 
conventional  methods  of  analysis,  Part  of  its  strength  oomes  from  its  solid 
mathematical  foundation,  and  future  development  will  involve  further 
exploration  of  the  underlying  mathematios  to  Identify  useful  results  which 
Inorease  the  method's  applicability. 
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ABSTRACT 

A  proeadura  for  comparing  thock  tantltlvlty  o*f  two  axplotivaa  -for 
•mail  tampla  tlsat  it  datcribad.  In  th i *  context,  thock  tantltlvlty 
rafart  to  tha  probability  of  datonatlon  whan  an  axplotiva  unit  l» 
axpotad  to  a  thock  ttlmulua.  Tha  comparlton  batwaan  axplotiva 
tantltivltlaa  It  mada  by  tatting  tha  hypothatit  that  tha  IQOpth 
par can 1 1 1  a  of  tha  population  of  critical  thock  valuat  for  a  ttandard 
axplotiva  it  graatar  than  tha  lOOpth  parcantila  of  critical  thock 
valuat  for  an  experimental  axplotiva.  Tha  loglatle  function  It 
attumad  at  tha  dlttribution  function  for  both  axplotivet,  and  aquality 
of  varlanca  <paral lal Itm)  batwaan  dlttr Ibutlont  naad  not  axltt.  Tha 
proeadura  utilliat  thraa  flxad  lav, alt  of  ttlmulut  to  whleh  axplotiva 
unltt  ara  axpotad  and  a  quanta!  retponte,  datonatlon  or 
non-detonation,  obtarvad  for  aaeh.  Monotonlxed  and  truncatad 
attlmataa  of  tha  datonatlon  probabl 1 1 1 i am  ara  utad  to  computa  minimum 
logit  ehi-tquara  attlmataa  of  tha  logistic  function  paramatart  for 
aach  axplotiva.  Tha  taat  it  battd  on  tha  attlmatad  diffaranca  batwaan 
tha  lOOpth  parcantllat  of  tha  two  axplotiva*.  Monta  Carlo  timulation 
hat  baan  utad  to  avaluata  tha  dltcr tmlnatory  powar  of  tha  proeadura 
ovar  a  wlda  ranga  of  paramatric  valuat.  Examplat  ara  glvan  for 
comparltont  mada  at  tha  20th  and  10th  parcantllat. 


1.  INTRODUCTION 

During  tha  davalopmant  of  axparimantal  axplotiva*  for  gun-flrad 
projactllati  on*  of  tha  critical  concarnt  it  tha  tantitlvity  of  an 
axplotiva  to  thock.  Shock  in  th i t  tanta  rafart  to  tha  rapidly 
davalopad  prattura  that  tha  axplotiva  taat  whan  It  it  loadad  Into  a 
proj ac 1 1 1 •'  and  flrad  from  a  gun.  If  tha  axplotiva  it  tantitlva  to 
this  typa  of  thock,  a  pramatura  datonatlon  may  occur.  Tha  axplotiva 
will  datonata  whila  tha  projactlla  It  still  in  or  jutt  balng  axpallad 
from  tha  gun  barral .  Thla,  of  courte,  It  a  vary  hazardoua  and 


undesirable  situation. 


In  order  to  test  explosives  for  shock  sensitivity,  a  prematura 
simulator  tast  facility  Mat  fabricated  at  NSWC,  Dahlgren,  Virginia. 
The  simulator  essentially  consists  of  a  1000  pound  drop  vehicle  which 
can  accommodate  a  scaled  down  projectile  loaded  with  explosive.  The 
test  is  conducted  by  dropping  the  vehicle  and  projectile  from  a 
predetermined  height  onto  a  heavy  steel  plate.  At  Impact  a  plunger 
is  forced  into  the  projectile  thus  simulating  the  rapidly  increasing 
pressure  that  occurs  In  conventional  gun  systems.  The  test  result  is 
either  a  detonation  or  non-detonation.'  For  each  test  unit  the 

•  .**  t> 

observed  outcome  is  associated  with  a  given  drop  height.  This  kind  of 

t 

data  has  been  collected  for  standard  explosives  which  are  used  in  Navy 

I 

projectiles.  Newly  developed  experimental  explosives  are  Also  tested 
for  shock  sensitivity  In  this  manner.  A  requirement  exists  for  a 
procedure  which  can  collect  and  analyze  this  data  to  determine  If  a 
given  experimental  explosive  Is  less  sensitive  than  a  standard 

explosive. 

2.  EXPERIMENTAL  CONDITIONS  AND  ASSUMPTIONS 

The  experimental  conditions  and  assumptions  under  which  this 

comparison  will  be  made  are  as  followsi 

a.  The  response  is  a  quantal-type  response |  that  Is,  there  is 

either  a  detonation  or  no  detonation.  This  quantal  response  Is 

related  to  the  drop  height  stimulus  level  by  some  probability 

distribution.  This  probability  distribution  governs  the  probiMl  Ity 
of  detonation  for  all  drop  heights  for  a  particular  explosive. 

b.  Knowledge  of  the  standard  explosive  is  assumed  both  with 
respect  to  the  type  of  distribution  and  also  the  value  of  the 
distribution  parameters.  This  information  will  have  been  gained 


through  sufficient  testing  of  the  standard  explosive. 

c.  The  experimental  explosive  is  an  unknown.  For  the  purposes  of 
the  present  discussion  we  shall  assume  that  it  has  the  same  type  of 
probability  distribution  as  the  standard.  However,  the  robustness  of 
this  assumption  is  a  question  which  can  be  investigated  under  the 
proposed  comparison  procedure.  The  values  of  the  parameters  of  the 
experimental  distribution  are  unknown. 

d.  The  number  of  test  units  of  the  experimental  explosive  is 
severely  restricted.  The  number  of  units  available  will  be  between  10 
and  IS.  This  restriction  arises  mainly  due  to  a  high  cost  per  test 
and  the  large  number  of  test  conditions  that  need  to  be  evaluated. 

e.  The  measure  of  sensitivity  is  defined  as  the  lOOpth  percentile 
of  the  probability  distribution  for  a  particular  explosive,  where 
0<p<.5.  That  is  the  value  of  drop  height  (stimulus  level)  at  which 
lOOp  percent  of  the  population  of  experimental  units  would  detonate. 
The  smaller  the  value  of  the  lOOpth  percentile,  the  more  sensitive  Is 
the  explosive. 

f.  The  comparison  procedure  should  protect  against  selecting  an 
experimental  explosive  over  the  standard  explosive  unless  the  data 
indicates  with  high  probability  that  the  experimental  is  less 
sensitive  than  the  standard. 

3.  INITIAL  EFFORTS 

A  computer  program  was  written  whereby  sensitivity  test  data  could 
be  simulated  and  analyzed  under  certain  distributional  assumptions  and 
for  particular  data  collection  designs.  A  normal  distribution  was 
assumed  as  the  governing  probability  dlstrih  M  n.  Available  data 
provided  no  evidence  to  seriously  question  this  assumption.  Maximum 
likelihood  estimation  was  used  to  estimate  the  parameters  of  the 


distribution  for  tht  experimental  explosive. 

Three  data  collection  designs,  th*  up-and-down,  Langli*  and  probit 
designs  warn  evaluated  for  sample  six**  of  10  and  20.  A  britf 
description  of  each  design  is  given  below.  Hor*  detailed  information 
can  be  found  in  references  4,  5  and  6, 

Th*  up-and-down  design  consists  of  choosinQ  a  starting  height  and 
test  increment.  Then  th*  stimulus  level  it  raised  or  lowered  by  th* 
amount  of  th*  test  increment  according  to  whether  a  non-detonation  or 
a  detonation,  respectively,  was  obtained  on  th*  previous  test. 

The  Lanol ie  design  requires  choosing  two  heights  initially,  on* 
at  which  no  test  units  are  expected  to  detonate  and  on*  at  which  all 
test  units  are  expected  to  detonate,  The  first  stimulus  level  Is  the 
average  of  these  two  heights.  Subsequent  heights  are  chosen  based  on 
tho  type  and  sequence  of  outcomes  from  previous  tests.  Accordingly, 
th*  next  stimulus  level  is  obtained  by  counting  bacKwards  until  a 
prior  stimulus  level  is  found  which  has  inclusively  encompassed  an 
equal  number  of  detonations  and  non-detonations.  Th*  average  of  this 
stimulus  level  and  th*  most  recent  stimulus  level  becomes  the  next 
level.  If  no  such  prior  level  can  be  found  then  the  next  stimulus 
lovel  is  obtained  by  averaging  th*  most  recent  stimulus  level  with  th* 
upper  bound  or  lower  bound  depending  on  whether  th*  previous  outcome 
was  a  non-detonation  or  a  detonation,  respectively. 

Th*  or obi t  design  is  th*  simplest  of  th*  three  and  requires  only 
that  a  prescribed  number  of  stimulus  levels  are  chosen  an:  a 
designated  number  of  Items  are  tested  at  each  level.  Data  collection 
desigra  o.'  this  type  are  usually  associated  with  probit  analysis  which 
is  discussed  i.i  reference  5. 

Th*  maximum  likelihood  estimation  procedure  requires  that  a  zone 


of  mixtd  rtsults  <ZMR)  txist*  in  th*  data  btfort  tstlmat**  of  th* 
paramtttrs  of  th*  distribution  can  bt  aatimattd.  A  ZMR  it  simply  an 
inttrval  on  tht  stimulus  seal*  dtflntd  by  tht  maximum  non-dttonation 
and  tht  minimum  dttonation  whtr*  tht  formtr  txcttda  tht  latttr.  Tablt 
1  show*  tht  ptretntagt  of  timulattd  tualuatlona  of  a  total  of  400 
which  actually  yitldtd  ZMR'*  op  usablt  data  undtr  optimal  dttign 
condition*. 

Ml  A 

Probability  of  ZMR 

U  Up  4  Down  LiPflUt,  ECflfcJLi 

10  65*/.  73H  6W, 

20  90X  95X  93X 

For  tht  probit  dtslgn  tht  valutt  of  N  wtrt  actually  9  (3  obttrvation* 
at  tach  of  3  stimulus  ltvtls)  and  21  <3  obttrvation*  at  tach  of  7 
stimulus  Itvtlt).  As  ont  can  ttt  tht  ptretntagt  of  Hitt  that 
tstimatts  wtrt  not  obtainablt  rangt*  from  40X  to  2SK  for  a  samplt  slit 
of  10.  A  samplt  sizt  of  20  ptrformt  somtwhat  bttttr.  Howtvtr,  tht 
figurts  shown  art  for  optimal  choicts  of  dttign  paramtttrs.  Tht 
choicts  for  thtst  dtslgn  valuts  mutt  bt  mad*  with  rtsptct  to  tht 
unknown  paramtttrs  of  tht  distribution  for  tht  txptrimtntal  txplosiv*. 
This  makts  an  optimal  choiet  unllktly.  For  ltss  than  optimal  choicts 
of  tht  dtslgn  paramtttrs  tht  ptretntagt  of  tlmt  a  ZMR  Is  obtaintd  may 
go  at  low  as  40H .  This  it  unacctp tablt. 

4.  PRQP08ED  PROCEDURE 

Battd  on  tht  prtetding  rtsults,  wt  turntd  to  a  difftrtnt 
distribution,  tht  logistic  function,  and  a  difftrtnt  tttimatlon 
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tachniqu*  which  can  b«  taslly  appliad  whan  using  a  problt  dasign.  Tha 
logistic  function  cloaaly  approxlmatas  tha  normal  distribution  and,  as 
statid  in  rafaranca  5,  thara  is  littia  dlffaranca  batwaan  conclusions 
drawn  from  an  analysis  basad  on  aithar  distribution.  Tha  cumulattva 
distribution  function  is 

p  ■  I/O  ♦  axp<-  a  -  8  h)), 

whara  a  and  3  ara  distribution  paramatars,  x  raprasants  tha  oalua  of 
tha  stimulus  lawal  <haight>,  and  p  is  tha  probability  of  datonatlon 
for  a  tast  projactlla  droppad  from  halght  x.  Tha  function  It  aaslly 
llnaarizad  by  takinQ  tha  logit  p  ■  ln<p/q>,  whara  q  ■  i-p  to  obtain 

L  »  logit  p  ■  ln<p/q>  ■  a  +  6  x* 

Tha  paramatars  of  this  distribution  can  baastimatad  using  tha  minimum 
logit  chl-squara  tachnlqua.  Tha  probit  dasign  Is  particularly 
appropriata  for  this  astlmatlon  procadura,  aspacially  for  tha  small 
sqmpla  siza  rastrlctlon.  Tha  lOOpth  parcantila  can  than  ba  astlmatad 
by  substitution  In  tha  llnaarizad  form  and  solving  for  >•  to  gat 
x<p)  ■  lOOpth  parcantila  ■  <ln<p/q>  •'«>/$. 

This  application  was  discussad  by  J.  Barkson  in  rafarancat  2  and  3. 

Tha  comparison  hypothasls  Is  statad  as  followsi 

Hi  Tha  lOOpth  parcantila  of  tha  standard  axplosiva  is  graatar 
than  or  aqual  to  tha  lOOpth  parcantila  of  tha  axparimantal 
axplosiva. 

Accaptlng  H,  of  coursa,  Imp  1 i as  that  ena  concludas  tha  standard 
axploslua  Is  lass  sansitlva  than  tha  axparimantal  axplosiva.  Tha 
altarnatlva  hypothasls  isi 

H(a)i  Tha  lOOpth  parcantila  of  tha  standard  axplosiva  Is  lass 
than  tha  lOOpth  parcantila  of  tha  axparimantal  axplosiva. 

Tha  altarnatlva  hypothasls  is  accaptad  If  H  Is  rajactad  and  this 
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conclusion  implies  that  tht  standard  explosive  is  nor •  sensitive  than 
tha  exptr  imental  explosive.  Ut  assume  that  tha  distribution 
parameters,  “s  and  0S  ,  for  tha  standard  explosive  art  known. 
Therefore,  tha  lOOpth  parcantlU  for  tha  standard  distribution!  8<p> 
can  ba  calculatad. 

A  number,  n  (lass  than  or  equal  to  5) ,  of  exper Imental  explosive 
units  ara  tastad  at  aach  of  thraa  aqual 1 x  spacad  Haights  making  tha 
total  sampla  slza,  N,  aqual  to  3n.  An  estimate  of  probability  of 
datonation,  p,  is  obtalnad  at  aach  halght  by  dividing  tha  number  of 
datonations  at  that  haight  by  n.  Thata  estimates  ara  monotonized  to 
ba  non-decreasing  thus  satisfying  tha  requirement  of  a  non-decreasing 
eumulatiua  distribution  function.  Monotenizatloni  In  this  raspact)  Is 
diseussad  In  rafaranca  1.  Probability  astlmatas  of  1  and  0  may  occur 
bacausa  of  tha  small  sampla  alias  usad.  Whan  this  happans  a 
substitution  must  ba  mada  so  that  tha  logit  valua  can  ba  computad. 
Tha  valuas  of  .93  for  1  and  .03  for  0  ara  racommandad  as  truncated 
astlmatas  to  ba  substltutad  in  such  instances.  Thasa  valuas  wara 
chosan  based  primarily  on  an  empirical  rule  given  In  one  of  tha 
earlier  Berkson  papers  and  tha  fact  that  they  provide  a  workable 
result  for  tha  small  sampla  sizes  of  interest.  Estimates  of  and 
for  tha  experimental  explosive  can  than  ba  computad  using  tha  minimum 
logit  chi-square  techniques  given  by  Berkson  in  rafaranca  3.  Thasa 
estimates  ara  designated  a  and  b,  respectively. 

If  b  is  non-zero  we  can  astimata  tha  tOOpth  percentile,  E(p),  of 
the  distribution  for  tha  axparlmantal  explosive  by  substitution  as 
shown , 

E(p)  *  <ln<p/q)  -  a)/b. 

Tha  estimated  difference  between  tha  lOOpth  percentile  for  the 


atandard  axploalva  and  tha  lOOpth  parcantila  for  tha  axpar Imantal 
axploalva  it  glvan  by 

D  «  ?<p>  -  E(p> • 

Tha  varianca  of  th i *  dlffaranca  la  glvan  by 

a2  -  U/b23  t<l/Z  nw)  4  <0 n  nw<x  -  TO2  1, 

whara 

w  «  p<l-p>,  x  ■  drop  halght,  V  ■  I  nwx/E  nw  and  0  ■  Of  -  E<p>>2  . 
Tha  aummatlon  la  takan  ovar  tha  thraa  valuaa  of  halght.  14a  ahall 
rajact  H  If  D  la  laaa  thin  or  aqual  to  e«a  whara  c  la  choaan  by  tha 
axparlmantar  to  glva  an  iceaptabla  probability  of  rajacting  H  whan  It 
la  actually  trua«  Thla  la  a  Typa  1  arror  and  for  tha  atatad  H  la 
aqulvalant  to  rajacting  tha  atandard  axploalua  whan  It  la  actually 
laaa  aanaltlva  than  tha  axpar Imantal  axploalva.  Tha  valua  of  c  la 
choaan  to  ba  nagatlua  bacauaa  wa  want  to  rajact  H  with  low 
probability  If  H  la  actually  trua.  If  D  la  graatar  than  c«i|  all 
poaalbla  taat  out com* a  with  mora  datonatlona  at  tha  taat  laual  of 
graataat  atlmului  muat  ba  conaldarad  bafora  H  can  ba  accaptad.  If  any 
laad  to  a  rnjactlon,  H  muat  ba  rajactad*  '  Tha  ratlonala  for  thla  la 
that  If  any  of  thaaa  mora  aanaltlva  outcomaa  ylald  a  rajactlon  of  H 
than  laaa  aanaltlva  outcomaa  ahould  not  contradict  that  raault  by 
accapting  H. 

Flgura  1  la  an  llluatratlon  of  tha  comparlaon  proeadura  for  tha 
caaa  whara  b  la  non-zaro.  Tha  atlmulua  lavala  hava  baan  acalad  In 
approxlmata  unlta  of  tha  atandard  divlatlon  of  tha  dl atr I  but  I  on  for 
tha  atandard  axploalva.  Tha  axparlmantal  axploalva  waa  taatad  at 
atlmulua  lavala  of  l ,  2  and  3.  8< . 2>  and  eC2>  rapraaant  tha  20th 

parcantlla  and  tha  aatlmata  of  tha  20th  parcantila  for  tha  atandard 
and  axparlmantal  axploalvaa,  raapact I valy. 
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An  alternative  procedure  I*  required  wh*n  b  ■  0  tine*  we  then 
cannot  eatimate  a  lOOpth  percentile  -for  tha  experimental  exploalve. 
Conaequently,  we  aaaurne  all  axparlmantal  teata  ara  conductad  at  tha 
lowaat  of  tha  thraa  tait  lavala.  Tha  lowaat  taat  level  la  tha  1  aval 
of  leaat  atlmulua  and,  therefore,  tha  laval  with  tha  laaat  probability 
of  datonatlon.  Thla  it  a  conaervative  pos  1 1 1  on  In  tarmt  of  tha 
axparlmantal  exploalve.  If  all  taitt  had  baan  dona  at  tha  lowatt  taat 
laval  (LTD ,  certainly  no  greater  numbar  of  datonatlona  would  hava 
oeeurrad  than  tha  numbar  actually  obtarvad.  Naxt  wa  eomputa  an 
aitlmatai  $,  for  tha  probability  of  datonatlon  at  thla  atlmulua  laval. 
An  uppar  95X  confidanca  bound  for  tha  probability  of  datonatlon  at 
thla  taat  laval  la  computed  and;  If  nacaaaary;  truncated  at  .99.  The 
logit  of  thla  uppar  confidence  bound  la  calculated  and  daalgnatad  at 
tha  logit  truncated  bound  or  LTB.  Tha  LTB  value  la  aubatltutad  Into 
tha  logit  function  for  tha  atandard  exploalve  to  eetlmate  an  uppar 
bound  for  an  equivalent  atlmulua  level;  8<p>,  required  by  tha  atandard 
exploalve.  If  tha  difference  between  thla  equivalent  atlmulua  level 
and  tha  lowaat  teat  level,  0  ■  8<p>  -  LTL,'  la  laaa  then  K,  than  H  la 
rejected.  Aa  with  c,  k  la  choaan  aa  a  negative  value  In  order  to 
achieve  an  acceptable  probability  of  rejecting  H  whan  It  la  true.  Tha 
value  of  k  mutt  be  adjuatad  in  magnitude  whan  tha  value  of  tha  lowaat 
taat  laval  changaa. 

Figure  2  Uluutratee  graphically  tha  procedure  for  b  ■  0.  Thla 
example  la  for  a  daalgn  with  taat  atlmulua  lavala  of  1,  2  and  3  and 
tha  atlmulua  lavala  hava  bean  acalad  aa  before.  LTL  rapraaanta  tha 
lowaat  taat  level  and  8<p)  rapraaanta  an  aatlmata  of  an  equivalent 
atlmulua  level  for  tha  atandard  exploalve. 
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5.  EVALUATION  OF  PROPOSED  PROCEDURE 

Computtr  simulation  was  ussd  to  sualuats  ths  proposed  proctdurt. 

A  computer  mods!  was  wrlttsn  to  generate  samp  It  data  from  an 
underlying  logistle  distribution  for  ths  experimental  explosive.  The 
simulated  data  are  then  compared  to  a  nominal  standard  explosive  using 
the  proposed  procedure.  The  comparison  procedure  determines  an 
acceptance  or  rejection  of  the  hypothesis  H.  By  replicating  thl« 
process  many  times  one  can  obtain  an  intimate  of  the  probability  of 
accepting  H  under  a  given  set  of  distributional  assumptions. 

The  main  objective  of  the  evaluation  was  to  demonstrate  the 
feasibility  of  the  proposed  comparison  procedure  and  determine  the 
acceptance/rejection  behavior  over  a  wide  range  of  distributional 
assumptions  for  ths  experimental  explosive.  The  distribution  for  the 
nominal  standard  explosive  was  chosen  to  be  a  logistic  distribution 
with  ag  ■  0,  Bg  ■  1.468,  These  parametric  values  were  chosen  because 
they  matched  a  logistic  distribution  to  a  standard  normal  distribution 
(mean  ■  0,  variance  ■  1)  at  minus  one-sigma  and  plus  one-sigma 
abscissa  values.  This,  in  effecti  provided  a  standardization  with 
respect  to  the  parameters  of  the  standard  explosive.  In  order  to 
pursue  the  evaluation  systematically  with  respect  to  possible 
experimental  distributions,  two  relative  quantities,  R  and  delta,  were 
defined.  R  represents  the  ratio  of  @s  to  and  Is  equivalent  to  the 
ratio  of  the  standard  deviation  of  the  experimental  explosive  to  the 
standard  deviation  of  the  standard  explosive.  Experimental 
distributions  with  values  of  R  from  .5(.9>2(1>4  were  chosen  to  cover  a 
wide  range  of  relative  variation.  Delta  represents  the  median  of  the 
experimental  distribution  minus  the  median  of  the  standard 


convenient  meant  of  relating  an  attuned  experimental  dittribution  to 
the  ttandard  dittribution  with  reepect  to  location.  For  each  value  of 
R  the  evaluation  wat  done  for  valuet  of  delta  from  -1<.5)2<1)5.  Thit 
generated  60  experimental  dl etr ibutlone  to  be  compared  to  the  ttandard 
dittribution. 

Two  probit  detlgnt  were  telected  for  comparlton.  The  flrtt  design 
uted  ttimuiut  level*  of  1,  2  and  3,  c  ■  -1.3  and  k  ■  -1.3.  The  tecond 
detign  uted  ttimuiut  levelt  of  3,  4  and  5,  c  ■  -1.3  and  k  ■  -2.3.  In 
termt  of  the  attumed  ttandard  dittribution,  the  ttimuiut  levelt  for 
the  flrtt  detign  corretpond  to  approximately  one-,  two-  and 
three-tigma  unit*  above  the  median  value,  whereat  the  ttimuiut  levelt 
for  the  tecond  detign  corretpond  to  approximately  three-,  four-  and 
five-tlgma  unite  above  the  median  value.  Thete  detlgnt  were  ehoten 
with  the  idea  of  "overteit Ing"  the  experimental  explosive.  Thit  wat 
an  attempt  to  provide  additional  protection  agalntt  rejecting  the 
ttandard  explotive  unlett  the  experimental  expletive  were  clearly  lett 
tentitive.  The  valuet  of  c  and  k  were  determined  empirically  by 
running  the  eimu1atinr>  with  arbitrary  trial  valuet.  Both  detlgnt  were 
evaluated  with  p  »  .2  for  all  60  combination!  of  R  and  delta. 
Additionally,  the  firtt  detign  wat  alto  evaluated  with  p  ■  .1  for  all 
combination!  of  R  and  delta.  The  evaluation!  were  done  with  n  ■  3  and 
10  which  corretpond  to  total  tample  tiiet  of  15  and  30,  retpect I vely. 
The  probability  of  acceptance  of  H  wat  ettlmated  for  a  particular 
condition  by  replicating  that  condition  800  timet.  Thit  number  of 
replication!  enturet  that  the  ettlmated  probabl 1 i ty  It  within  .03  of 
the  true  probability  with  93*/.  confidence. 
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4.  RE8ULT8 

Th*  results  ar#  shown  In  Figures  3-11.  For  both  sample  sizes 
«4Ch  figure  displays  th#  plot  of  estimated  probability  of  accepting  H 
versus  delta  for  a  particular  design,  R-valu#  and  p-valu*.  Figures 
3-9  and  4-8  show  th*  results  for  th*  flrat  dttlgn  for  p-values  of  .2 
and  .1,  r*apectio*1y.  Figur*»  9-11  show  th*  rtaulta  for  th*  second 
d*aign  for  p  -  .2.  Only  th*  rtaulta  for  R-ualu*»  of  .9,  1  and  A  ar* 
ah own  In  th*  lnt*r*at  of  tawing  tpac*.  Th*t*  R-walu**,  however, 

encompass  th*  rang*  ut*d  In  th*  evaluation  and  ar*  sufficient  to  show 
th*  effect  of  relative  warlatlon  upon  th*  *«tlmat*d  probability. 

A*  stated  abow*  delta  Is  th*  median  of  th*  probability 
distribution  for  th*  experimental  explosive  minus  th*  median  of  th* 
probability  distribution  for  th*  standard  exploslw*.  For  an 
experimental  exploslw*  with  specified  variability,  a  decrease  In 
sensitivity  Is  associated  with  an  Increase  In  delta.  A  comparison 
procedure  based  on  sensitivity,  as  defined  previously,  should  yield  a 
decrease  In  th*  probability  of  accepting  th*  standard  explosive  as 
delta  Increases.  Also,  a  particular  value  of  delta  exists  below  which 
H  Is  true  (the  experimental  explosive  Is  more  sensitive)  and  above 
which  H  Is  false  (the  standard  explosive  Is  more  sensitive).  Th* 
probability  of  accepting  th*  standard  explosive  should  go  from  one  to 
aero  at  this  value  of  delta,  of  course,  for  an  Ideal  comparison 
procedure.'  These  critical  values  of  del ta  are  .given  In  Table  2  for 
th*  Indicated  values  of  R  and  p. 
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TABLE  2 


CM-Hsil  Bt.Uu 


JL 

pi.l 

d*.2 

0.3 

-.66 

-.42 

1.0 

0.00 

0.00 

4.0 

3.93 

2.49 

Tht  clottnttt  to  which  the  proetdurt  approtchtt  tht  idtal  varlta 
■for  tht  llluttrattd  txampltt.  Tht  capability  to  dltcrlminatt  bttwttn 
txplotivta  of  dlffirtnt  mtns 1 1 1 v I ty  It  rtaaonably  prtcltt  for  tht 
iltuatlont  whtrt  tht  variation  of  tht  txptrlmtntal  txplotlvt  It  Ittt 
than  or  tqual  to  tht  variation  of  tht  ttandard  txplotlvt,  that  It, 
whtn  R  *  .9  and  1.  Thlt  It  thown  In  Flgurtt  3,  4,  6  and  7  for  tht 
flrat  dttlgn  and  Flgurtt  9  and  10  for  tht  ttcond  dttlgn.  Tht 
probability  of  acctptlng  tht  ttandard  txplotlvt  dropt  quickly  at  tht 
ttntl t I v 1 ty  of  tht  txptrlmtntal  txplotlvt  Ittttnt,  that  It,  at  dtlta 
inertattt.  Whtn  tht  uarlatlon  of  tht  txptrlmtntal  txplotlut  txcttdt 
tht  variation  of  tht  ttandard  txplotlvt,  that  It,  whtn  R  ■  4,  tht 
dlterlmlnalory  capability  It  much  Ittt  prtcltt  for  both  dttlgnt  at 

i 

thown  In  Flgurtt  3,  8  and  11.  Htrt  tht  probability  of  acctptlng  tht 
ttandard  txplotlvt  Ittttnt  vtry  gradually  at  dtlta  (nertattt. 

Btcautt  of  tht  ttvtrt  conttqutnett  of  Ineorrtetly  rtjtctlng  tht 
ttandard  txplotlvt  <Typt  I  trror),  tptclflc  mtaturttv  wtrt  Incorporattd 
to  rtduet  tht  probability  of  thlt  happtnlng.  Howtvtr,  at  a  rtault  of 
rtduclng  tht  probability  of  a  Typt  1  trror,  tht  probabl 1 1 ty  of 
committing  a  Typt  11  trror  It  Incrtattd.  In  thlt  conttxt  a  Typt  11 
tmor  mtant  acctptlng  tht  ttandard  txplotlvt  whtn  tht  txptrlmtntal 
txplotlvt  It  actually  lttt  atntltivt.  Thlt  tfftet  It  eltarly 
dtmonttrattd  In  Flgurt  3.  In  thlt  catt  tht  ttandard  txplotlvt  and  tht 


axparlmantal  axploflva  art  of  equal  etna  1 1 i v I ty  whan  delta  it  -.42. 
Thi*  it  the  critical  value  of  dtlta  from  Tabl*  2  for  R  ■  .5  and  p  ■ 
.2.  For  delta  lata  than  -.42,  H  if  tru*.  That  If,  th«  ftandard 
axploflva  l«  lttf  eenfltlve  than  the  experimental  axploflva  and  fhould 
ba  accaptad.  Tha  probability  of  doing  that  If  naar  1  for  dalta 

tmallar  than  -.42.  At  dalta  Ineraatat  to  .5  (tha  axparlmantal 
axploflva  bacomee  laff  fentltlve  than  tha  ftandard),  tha  probability 
of  accaptlng  H  ramalnt  high.  Llndar  thaia  condltlona  a  Typa  II  arror 
would  ba  vary  likaly  bacauta  H  If  not  trua  but  would  ba  accaptad  with 
high  probability,  In  tarmt  of  axploflvat,  thlf  meanf  tha  ftandard  If 
mora  fane  1 1 1 ua  than  tha  axparlmantal  but  would  ba  accaptad  with  high 
probability,  Howavar,  thlf  kind  of  arror  If  nut  at  aarlouf  at 
Ineorractly  rajacting  a  laff  aantttlua  ftandard  axploflva  In  favor  of 
accaptlng  a  mora  aantltiva  axparlmantal  axploatva  (Typa  1  arror).  Tha 
prlea  for  keep  I ng  tha  probability  of  a  Typa  I  arror  fmall  it  accaptlng 
an  Incraaaa  In  tha  probability  of  a  Typa  II  arror,  For  both  daolgna 
tha  daalrad  fmall  probability  of  a  Typa  1  arror  If  attalnad  for  R  ■  .5 
and  1|  and,  aa  axpaetad,  thlf  it  aecompanlad  by  a  relatively  larga 
probability  of  a  Typa  II  arror.  Additionally,  tha  larga  probability 
of  a  Typa  II  arror  axtandf  ovar  a  graatar  ranga  of  dalta  for  tha 
facond  daflgn.  Thlf  If  an  affact  of  tha  Incraaaad  ttlmuluf  valuat 

utad  In  tha  facond  daflgn,  Whan  tha  variation  of  tha  axparlmantal 
axploflva  baeomaa  relatively  large,  that  If,  whan  R  ■  4,  neither 
daflgn  produeef  email  probabll Itlaa  of  a  Typa  I  arror,  Thlf  If 
particularly  trua  aa  delta  approaehee  tha  critical  value.  Thlf  wae 
noted  earlier  In  tha  commentf  concerning  praclea  dltcr Imlnatlon 
between  explotlvee  with  different  aanel tlvltlae.  Thlf  If  eauaed  by 
tha  relatively  large  variability  of  tha  axparlmantal  axploflva. 


8mallar  valuas  of  c  and  k  would  dtcraaaa  thia  probability  of  a  Typa  1 
trror  but  tha  pracision  of  tha  discriminatory  capability  would  ramaln 
low.  Dacraaaat  in  c  and  k  may  causa  an  Intolarabla  incraase  in  tha 
probability  of  incorractly  accapting  tha  standard  axplosiva  for 
R-valuas  of  .9  and  1 • 

Tha  two  valuas  of  p  usad  with  tha  first  daslgn  provlda  an 
opportunity  to  obsarva  tha  affact  of  tha  walua  of  p  upon  t  tha 
comparison  procadura.  Tha  probability  of  accapting  H  for  p  ■  .2  is 
aqual  to  or  slightly  lowar  than  tha  probability  of  accapting  H  for  p  ■ 
.1  across  all  valuas  of  dalta.  This  maans  that  for  dalta  lass  than 
tha  critical  valua,  tha  probability  of  wrongly  rajacting  tha  standard 
axplosiva  Is  graatar  for  p  ■  .2.  Whan  dalta  is  graatar  than  tha 

critical  valua,  tha  probability  of  wrongly  aecapt I ng  tha  standard 
axplosiva  Is  graatar  for  p  ■  .1.  A  comparison  using  tha  smallar  valua 
of  p  would  ba  of  mora  Intarast  and  conslstant  with  making  tha  Typa  1 

arror  probability  small.  Howavar,  choosing  ton  small  a  valua  of  p  may 

* 

rasult  in  a  lass  praclsa  comparison.  Additional  simulations  would  ba 
naadad  to  offar  mora  undarstanding  of  this  situation. 

Comparing  tha  simulation  rasults  for  tha  two  sampla  slzas  shows 
that  for  most  valuas  of  dalta  tha  Typa  I  and  Typa  II  arror 
probabl 1 1 tlas  ara  lass  for  n  ■  10  than  for  n  ■  9.  This  would  ba 
axpactad  sinca  tha  rajactlon  valuas  of  c  and  k  wara  hald  constant  for 
both  sampla  alias.  Howavar,  soma  axcapttons  do  axlst  for  cartain 
ragions  of  dalta.  Thasa  axcaptions  ara  also  dapandant  upon  tha  valua 
of  R.  Idaally,  tha  probability  of  accapting  tha  standard  axplosiva 
would  ba  graatar  for  n  ■  10  than  for  n  ■  9  whan  dalta  is  lass  than  or 
aqual  to  tha  critical  dalta.  For  dalta  valuas  graatar  than  tha 
critical  valua  tha  ravarsa  would  ba  trua.  Tha  probability  of  Typa  I 


and  Typ*  11  error*  would  then  be  let*  ■for  th*  larger  tempi*  *1**  in 
til  c«m>  On*  ctn  not*  that  for  R  ■*  .3  and  1  th*  curvts  for  n  ■  10 
croft  fallow  th*  curve*  for  n  ■  3  aft*r  th*  critical  value  of  dalta  hat 
b**n  r*ach*d.  Thia  mean*  that  for  dalta  b*tw**n  th*  critical  valu* 
and  th*  crott-over  point  th*  probability  of  a  Typ*  11  *rror  it  gr*at*r 
for  n  *  10  than  for  n  *  3.  Wh*n  R  ■  4  and  th*  variation  of  th* 
•xp*r Imenttl  expletive  It  1arg*r  than  th*  variation  of  th*  ttandard( 
th*  two  curv*t  erott  b*for*  th*  critical  valu*  of  d*lta.  H*r*  th* 
probability  of  wrongly  r*J*cting  th*  ttandard  explotlv*  <Typ*  1  *rror> 
it  greater  for  n  ■  10  than  for  n  ■  3  when  delta  lies  between  th* 
crott-over  point  and  th*  critical  valu*.  Adjuttment  could  b*  mad*  on 
th*  value* '  of  c  and  k  to  reconcile  tom*  of  th*  apparent  anomall** 
between  th*  tempi*  tlz*t.  However i  th*  effect  of  any  adjuttment  it 
dependent  upon  th*  value  of  R>  If  adjuttmentt  are  mad*  to  force  a 
more  contittent  retpont*  for  R*  .3  and  1,  th*  anomaliet  are  tpread 
over  a  greater  rang*  of  delta  for  R  *  4,  and  vice  v*r»*.  Thl*  it  on* 
of  th*  difficult!**  impoted  by  th*  Initial  ground  rule*  which  allow 
for  difference*  In  th*  variation  for  the'  ttandard  and  experimental 
explotl vet. 


7.  SUMMARY 

Th*  propoted  procedure  offer*  a  methodology  for  comparing  an 
experimental  explotlv*  to  a  ttandard  explotlv*  with  retpect  to 

**n*ltlvlty  under  th*  ttated  ground  rule*.  Th*  procedure  It 
particularly  applicable  to  a  tltuation  where  a  1  in  I  ted  number  of 

experimental  t**t  unit*  are  available.  Th*  retult*  of  th*  tlmulated 
example*  demonttrat*  th*  dltcr Imlnatory  capability  of  th*  compariton 
procedure.  Th*  computer  model  written  to  tlmulat*  th*  procedure  can 
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be  used  tu  determine  optimal  design  parameters  with  regard  to 
probabll ities  of  Type  1  and  11  errors.  Alto,  the  simulation  modtl  can 
b»  used  to  avaluate  robustness  of  the  procedure  with  regard  to  the 
attumad  probability  distribution  of  tht  experimental  explosive. 

In  conclusion,  tha  propotad  procadura  appears  to  provide  a 
raatonabla  antwar  for  auary  set  of  tatt  outcomat.  Additionally,  by 
proparly  choosing  valuta  for  tha  dasign  and  tast  parameters,  tha 
raquirad  protactlon  against  Typa  1  and  II  arrors  can  ba  obtained. 
This  is  a  dacidtd  improvement  ovar  currant  comparison  proccdurts  baing 
usad  In  sansitivity  tasting  of  naw  explosives. 
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ABSTRACT! 

Tha  first  author  oonduotad  a  study  to  invaatigata  raolal  diffarenoea  In  aklnfold 
thloknass  aaasuramants  and  feha  oaloulatad  paroant  body  fat  for  normal  waight  and 
ovarwalght  soldlars.  It  was  hypothaalxad  that  thara  was  no  algnlfloant 
dlffaranoas  batwaan  aklnfold  maasuramanta  and  paroant  of  body  fat  batwaan  blaek 
and  whlta  soldlars.  Multivarlats  analysis  of  varlanoa  and  dlsorlmlnant  analysis 
was  usad  to  analyxa  tha  data  and  taat  tr»a  axparlmantal  hypothaaaa.  Dlffarant 
combinations  of  dapandant  variablas  wara  oonsldarad  in  tha  analysis  to 
Invaatigata  tha  oonslstanoy  of  tha  statistioal  daolslons  batwaan  and  within 
raolal  and  waight  groups.  This  study  shows  that  tha  multivariate  prooaduras 
usad  ylaldad  oonaistant  daolslons  for  various  combinations  of  tha  dapandant 
variablas. 

INTRODUCTION! 

Nutritional  asaassmant  using  aklnfold  oallpara  to  avaluata  laannass/fatnasa 
has  baan  usad  axtanalvaly  slnoa  tha  davalopmant  of  oalipars.  For  savaral  yaars 
tha  United  States  Amy  has  usad  skinfold  maasuramanta  to  aoraan  Its  members  for 
laannass/fatnasa  and  physical  fitness.  Whan  an  Individual  axoeads  tha  waight 
for  height  required  maximum,  that  parson  is  evaluated  on  tha  basis  of  a 
percentage  body-fat  standard. 

This  researoh  was  designed  to  oompara  and  avaluata  tha  aklnfold  measurements 
and  body-fat  paroant  determination  of  mala  soldiers.  It  was  hypothesised  that 
(a)  thara  Is  no  algnlfloant  dlffaranoa  between  aklnfold  measurements  taken  from 
blaok  soldlars  and  those  taken  from  whlta  soldlars,  and  (b)  thara  Is  no  signifi¬ 
cant  dlffaranoa  in  paroant  body  fat  between  blaok  soldiers  versus  white  soldiers 
as  determined  by  tha  established  method.  Skinfold  maasuramanta  wara  made  during 
routine  physioal  examinations  on  basio  trainees,  and  during  an  ovarwalght 
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soreening  of  oareer  soldiers.  Eaoh  group  was  divided  by  raoe  and  age. 

PROCEDURE: 

Subjects  for  the  study  were  male  soldiers  at  Fort  Banning,  Georgia,  between 
17  and  39  years  of  age.  The  purpose  and  procedures  involved  in  the  study  were 
explained  to  eaoh  soldier  before  the  participant  signed  a  volunteer  oonsent 
form.  Four  hundred  fifty-two  soldiers  volunteered  for  the  study.  However,  only 
302  soldiers  were  used  baoause  those  over  MO  years  of  age,  or  members  of  other 
ethnio  or  raolal  groups  were  not  inoluded  in  the  analysis. 

Subjeots  were  divided  into  two  groups:  Group  I  -  young  trainees  from 
Regular  Army,  National  Guard,  and  Reserve  units  who  had  just  oompleted  seven 
weeks  of  Infantry  basic  training  and  Group  IX  -  oareer  soldiers  from  an  Infantry 
brigade  who  had  been  identified  as  overweight  by  Army  height-weight  tables. 
Subjeota  for  Group  X  were  seleoted  at  random  during  a  routine  physioal  examina¬ 
tion.  The  number  of  eaoh  raoe  per  group  was  kept  equal  for  statistical 
comparisons,  Eaoh  group  is  subdivided  by  race.  Raoe  was  determined  visually, 
by  questioning  the  partiolpant,  and  from  medioal  history  forms.  The  composition 
of  eaoh  group  is  given  in  Table  1. 


TABLE  1.  Distribution  of  the  Subjeots  by  Raoe  &  Age 


Height  was  measured  in  stooklng  feet  with  feet  together,  back  and  heels 
against  the  upright  bars  of  a  height  soale,  head  in  the  Frankfort  horizontal 
plan  ("look  straight  ahead")  and  stand  erect  ("stand  up  straight").  All  weights 
were  reoorded  to  tha  nearest  quarter  pound . 

Four  skinfold  thiokness  measurements  were  taken  with  a  Lange  skinfold 
oaliper  oalibrated  to  exert  a  pressure  of  lOg/sq  mm  of  jaw  surfaoe.  All 
skinfold  measurements  were  taken  by  the  first  author.  All  measurements  were 
taken  on  the  right  side  of  the  body,  with  the  subjeot  standing.  The  sites 
seleoted  were  as  follow: 

1.  Bioeps:  over  the  midpoint  of  the  musole  belly  with  the  arm  hanging 
vertloally  at  rest. 


2.  Trioeps:  over*  the  midpoint  of  the  muscle  belly,  midway  between  the 
olecranon  and  the  tip  of  the  acromion,  with  the  arm  hanging  verti¬ 
cally  at  rest. 

3.  Subscapular  *.  Just  below  the  tip  of  the  inferior  angle  of  the 
scapula,  at  an  angle  of  about  45°  to  the  vartioal. 

4.  Supraillao:  Just  above  the  iliao  nrest  in  the  mid-axillary  line, 

A  reading  was  taken  and  reoorded  at  each  of  the  four  skinfold  sites  and  than 
repeated  twice  more  in  suooesslon.  Readings  were  taken  to  the  nearest  0.5mm. 
The  three  readings  at  eaoh  site  was  averaged  to  the  nearest  tenth  of  a 
Millimeter  and  the  average  values  totaled  to  give  the  sum  of  four  skinfolds  for 
obtaining  peroent  body-fat  value  from  the  Durnin  and  Womersley  table  (Reference 
2>. 


ANALYSIS  OF  DATA  (GENERAL); 

The  data  analysis  was  oonduoted  using  the  statistical  paokage  for  the  Social 
Soiences  (SPSS)  Update  7-9.  The  normal  plots  for  the  different  variables  in  tha 
study  all  had  linear  trends  indicating  normal  distributions.  The  test  for 
homogeneity  of  the  varlanoe/oovar lance  matrloles,  Wilk's  Lamda  and  Bartlett's 
test  of  sphericity  were  some  of  the  statistical  tools  used  in  the  analysis. 
Hypothesis  testing  was  oonduoted  at  the  5K  level  of  signifioanoe. 

|  RESULTS  AND  DISCUSSION; 

I 

1 .  Analysis  of  Group  1  (Normal  Weight  Group). 

The  results  of  the  data  analysis  for  the  normal  weight  group  are  shown 
in  Table  2  (next  page).  The  differences  are  marked  by  an  (*).  These 
differences  in  skinfold  measurements  are  examined  more  olosely  in  Table  3  below. 
The  dependent  variables  are  the  skinfold  measures  and  the  Independent  variable 
is  raoe  (i.e.,  blaok,  white). 


TABLE  3.  DIFFERENCES  BETWEEN  BLACK/WHITE 
NORMAL  WEIGHT  GROUP 


VARIABLE  UNIVARIATE 


F 

TRICEP 

25.6 

BICEP 

16.7 

SUBSCAPULAR 

1.2 

P 

MAN0VA  CORRELATION 
BETWEEN  DEPENDENT 

AND  CANONICAL  VARIABLE 

.000 

.80-i 

RELATIVE 

MAGNITUDE 

.000 

.64- 

REFLECTS 

UNIVARIATE 

.280 

.17 

RESULTS 

.002 

.49  - 

SUPRAILIAC 


9.5 


002 


TABLE  2.  MEASUREMENTS  FOR  NORMAL  WEIGHT  GROUP 


2,  Analysis  of  Group  2  (Overweight  Group). 


Similar  data  lor  the  overweight  group  are  shown  in  Tables  4  (next  page) 
and  5  below. 


TABLE  5.  DIFFERENCES  BETWEEN  BLACK/WHITE 
OVERWEIGHT  GROUP 


VARIABLE 

. 

UNIVARIATE 

F 

P 

MAN0VA  CORRELATION 
BETWEEN  DEPENDENT 

AND  CANONICAL  VARIABLE 

TRICEP 

20.4 

.000 

o94- 

RELATIVE 

MAGNITUDE 

BICEP 

6.2 

.015 

.52- 

REFLECTS 

UNIVARIATE 

SUBSCAPULAR 

3.6 

.087 

.36 

RESULTS 

SUPRAILIAC 

6.8 

.011 

.54 

3.  Summary. 

For  both  the  normal  and  overweight  groups  there  la  a  consistent 
difference  between  the  black  and  white  soldiers  with  respect  to  skinfold 
measurements  of  the  triceps,  biceps,  and  supralllao.  The  white  soldiers  have 
consistently  larger  measurements  on  these  variables. 
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4.  Excursion  1.  Consistency  of  MANOVA  Results  for  Normal  Wight  Group. 

The  consistency  of  the  MANOVA  for  detecting  differences  between  raolal 
groups  (blaok,  white)  and  identifying  the  variables  which  contributed  to  these 
dlffartnoes  (skinfold  measurements,  peroent  body  fat)  are  lndioated  in  Table  6. 
Note  that  the  results  are  oonslstenb  for  the  five  oomblnotlons  of  variables 
using  MANOVA  and  univariate  t-tests  (combination  d).  Combination  (e)  is  the 
MANOVA  that  considers  all  the  variables  examined  in  the  previous  combinations. 
The  variables  whioh  are  identified  as  contributing  to  the  differences  between 
the  raoes  for  the  normal  weight  group  are  completely  consistent  with  the 
findings  of  the  other  combinations  (a  through  d). 

TABLE  6 

EXCURSION  1:  MANOVA  CONSISTENCY  WITHIN  NORMAL 
_  ,,  WEIGHT  GROUP  -  BETWEEN  RACES 


VARIABLES 

CONSIDERED 

IN  MANOVA 

Z 

SPHERITY 

WILX'S 

X 

RELATIVE  MAGNITUDE  OF 
CORRELATION  BETWEEN 
DEPENDENT  AND  CANONI¬ 
CAL:  UNIVARIATE  F 

(a) 

Skinfold  only 

35.7 

0.000 

639 

0.000 

.848 

0.000 

TR1,  BICEP,  ILIAC 

(b) 

Ht,  Wt  only 

2.9 

.41 

79.1 

0.000 

.987 

.15 

No  Difference 

(c) 

a  and  b  above 

39.3 

.012 

881.1 

0.000 

.787 

0.000 

TRX,  BICEP,  ILIAC 

<d> 

%  Body  fat  and 

Age 

2  Separate 

T-Tesfcs 

%  Body  Fat 

(a) 

a,  b,  d  above 

161.5 

0.000 

1566 

0.000 

.780 

0.000 

TRI,  BICEP,  ILIAC 
%  Body  Fat 

NOTE:  (Statistlo/P  value) 


5.  Excursion  2:  Consistency  of  MANOVA  Results  for  Overweight  Group  Table  7 

The  same  findings  were  obtained  for  excursion  2  (see  excursion  1).  The 
same  variables  were  identified  as  contributing  to  differences  between  the  racial 
group,  and  the  results  were  consistent  for  all  combinations  of  factors 
considered  In  the  different  analyses  (a  through  d). 
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TABLE  7 


EXCURSION  2: 

MANOVA 

GROUP 

CONSISTENCY  WITHIN 
-  BETWEEN  RACES 

OVERWEIGHT 

VARIABLES 

CONSIDERED 

IN  HANOVA 

— / 

X 

SPHERITY 

WILK'S 

X 

RELATIVE  MAGNITUDE  OF 
CORRELATION  BETWEEN 
DEPENDENT  AND  CANONI¬ 
CAL:  UNIVARIATE  F 

(a) 

Skinfold  only 

7.6 

.714 

121.4 

0.000 

.761 

.001 

TRI,  BICEP,  ILIAC 

(b) 

Ht,  Wt  only 

3.6 

.320 

74.2 

0.000 

.989 

.674 

No  Differenoe 

(o) 

a  and  b  above 

18.2 

.737 

239.8 

0.000 

.754 

.003 

TRI,  BICEP,  ILIAC 

(d) 

%  Body  fat  and 

Age 

2 

Separate  T-Tests 

%  Body  Fat 

(e) 

a,  b,  d  above 

46 

.273 

450.1 

0.000 

.734 

.006 

TRI,  BICEP,  ILIAC 
%  Body  Fat 

NOTE:  (Statistic/P  velua) 


6.  Excursion  3:  raptorial  HANOVA. 

A  full  fsotorisl  MANOVA  was  oonduoted  on  the  Independent  variables  of 
raoe  (blaok,  white)  and  weight  group  (normal  weight,  overweight).  The  dependent 
measures  were  the  four  skinfold  measurements,  height,  weight,  age,  and  peroent 
body  fat.  The  results  were  entirely  consistent  with  the  findings  disouased  so 
far. 


Raoe  Effeot.  Wilk's  Lnmda  was  significant.  The  variables  that  contributed 
to  the  differences  between  races  were  again  identified  as  the  trioep,  bioep, 
supralliao,  and  peroent  body  fat. 

Weight  Group  Effeot,  As  expeoted,  Wilk's  Lamda  was  significant  on  all  the 
dependent  variables  exoept  for  height. 

Raoe  and  Weight  Group  Interaction.  The  interaction  effeot  was  not 
significant. 

7.  Excursion  4:  Dlaoriminant  Analysis  (All  Data  and  a  50%  Random  Cut) 

The  results  of  disorimlnant  analysis,  using  all  data,  and  classifying 
individuals  as  belonging  to  either  the  normal  weight  or  overweight  group  are 
shown  below  in  Table  8. 
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TABLE  8 


OROUPS  «  WEIGHTS  (NORMAL,  OVERWEIGHT) 

VARIABLES  a  TRICEP,  BICEP,  SUPRAILIAC,  RACE 
DATA  a  ALL  (N  a  302) 


ACTUAL 

GROUP 

NORMAL 

WEIGHT 


PREDICTED  MEMBERSHIP 


li  ii  f  mmmi 


OVERWEIGHT 


226 

193 

33 

85.4% 

14.6% 

76 

17 

59 

22.4% 

77.6% 

1.  CORRECT  CLASSIFICATION  a  (193  +  59)/302  a  83. MS. 

2.  REFERENCE:  AAKER,  C  max  a  Max  |  pr  Group  I,  Group  2^ 

a  Max  j  .748.  .252?  a  ,748 

3.  SINCE  .834  >  .748,  CLASSIFICATION  IS  ACCEPTABLE. 


Aooordlng  to  a  oritaria  by  Aakar  (rafaranoa  1),  tha  olaaaifioation  ia 
aooaptabla.  Nota  that  tha  variabla  uaad  wara  thoaa  datarminad  to  ba  aignlf leant 
oontributora  aa  datarminad  from  othar  analyaaa.  Tha  50%  random/  atrat^flad  out 
aohame  la  shown  in  Tabla  9. 

TABLE  9 

DISCRIMINANT  ANALYSIS 

50%  RANDOM  CUT  (STRATIFIED  BY  RACE,  WEIGHT  GROUP) 


NORMAL  WEIGHT 

■ 

226 

113 

(56  BLACK,  57  WHITE) 

OVERWEIGHT 

■ 

76 

38 

(19  BLACK,  19  WHITE) 

VARIABLES 

•  TRICEP,  BICEP, 

ILIAC,  RACE 

a  *  rVvo 


V/ 


The  results  of  the  disorlmlnant  analysis  based  on  a  single  50}  random 
out  and  classifying  the  remaining  data  not  used  to  oonstruot  the  discriminant 
functions  are  shown  in  Tabla  10.  Note  there  is  only  a  31  difference  in  the 
peroentage  of  oorreot  classification  when  oompared  to  the  results  using  all  data 
presented  in  Table  8. 


TABLE  10 


DISCRIMINANT  ANALYSIS  -  CLASSIFICATION  BASED 
ON  A  50}  RANDOM  CUT 


PREDICTED  MEMBERSHIP  OF 

ACTUAL 

REMAINING 

DATA 

GROUP 

N 

NORMAL 

OVERWEIGHT 

NORMAL 

113 

91 

22 

80.5} 

19.5} 

OVERWEIGHT 

38 

8 

30 

21.1} 

78.91 

1.  CORRECT  CLASSIFICATION 

a  n  ...u  _  rviio  >  d  a  a 

■  V  ■ 

.801 

2.  C  max  ■  .748  <  .801 


Brief  Summary:  This  single  oase  study  indicates  that  multivariate  procedures 
(MAN0VA,  disorlmlnant  analysis)  yield  oonsistent  decisions  for  various  combina¬ 
tions  of  data.  The  decisions  were  consistent  for  several  excursions  within  and 
between  groups.  Although  these  results  are  not  theoretically  oonoluslsve,  and 
represent  only  one  set  of  data,  they  add  oonfldenoe  to  drawing  oonolusions  from 
data  analysis  that  are  driven  to  answer  a  series  of  "what  if"  questions. 


REFERENCES 


1.  Aaker ,  David  (1971),  Multivariate  Analysis  in  Marketing;  Theory  and 
Application.  Wadsworth  Publishing  Company,  California. 


Durnin,  J.  and  Woraersley,  J.  (1974),  Body  fat  assessed  from  total  body 
density  and  its  estimation  from  skinfold  thiokness;  measurements  on 
481  men  and  women  aged  from  16-72  years.  Br.  J.  Nutr.  32,  77-97. 


3.  Hull,  C  (1901),  SPSS  Update  7-9.  MoGraw-Hill,  New  York. 


4.  Harris,  R.  (1975),  A  Primer  of  Multivariate  Statistics,  Aoademio  Press, 
New  York.  ~ 


nu 

h-V.' 

.wc*. 


DESERT  CAMOUFLAGE  PAINT  EVALUATION 
FOR  SAUDI  ARABIAN  NATIONAL  GUARD 


GEORGE  ANITOLE  AND  RONALD  L.  JOHNSON 

US  Army  Belvoir  Research  and  Development  Center 
Fort  Belvoir,  Virginia  22060-5606 


ABSTRACT 


This  i>aper  describes  the  procedure  for  selecting  candidate 
desert  paint  colors,  the  field  test,  and  statistical 
analysis  procedures  which  enabled  the  final  color  selection. 
Eleven  tactical  vehicles  were  painted  in  camouflage 
colors  and  viewed  by  ten  ground  observers  at  four  different 
color  backgrounds  in  Saudi  Arabia.  The  method  of  testing 
involved  a  ranking  technique  using  a  direct  comparison 
between  two  vehicles.  This  is  more  sensitive  and  discerning 
than  measurements  on  a  scale  of  values  and  overcomes  the 
problem  of  inconsistency  of  judgements  expressed  by  the 
same  observer.  The  observers  were  presented  with  every 
possible  combination  of  paired  vehicles  and  forced  to 
ahoose  between  them.  The  data  was  summarized  for  each 
observer  and  test  location  and  analyzed  statistically  to 
determine  preferred  colors  in  the  order  of  rank,  establish 
confidence  limits,  and  establish  color  groupings  for  each 
site. 
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1.0  INTRODUCTION 


In  April  1980,  the  Project  Manager  (PM),  Saudi 
Arabian  National  Guard  (SANG),  Modernization,  requested 
assistance  to  provide  camouflage  for  SANG.  A  fact  finding 
team  visited  Saudi  Arabia  in  September  1980  to  gather 
specific  information  required  to  develop  a  camouflage 
program  including  desert  colored  paints,  nets,  and  uniforms. 
This  paper  presents  the  paint  development  effort  and 
the  selection  of  the  final  camouflage  paint  colors  for 
SANG. 


The  color  used  on  SANG  vehicles  at  the  time  of 
the  faot  finding  visit,  resembled  a  battleship  gray, 
high  luster  paint.  This  color  contrasted  greatly  with 
the  Saudi  Arabian  desert  backgrounds,  and  the  vehicles 
could  be  seen  from  great  distances.  It  was  considered 
an  immediate  requirement  by  PM,  SANG  to  develop  a  desert 
colored  paint  suitable  for  the  Saudi  Arabian  deserts. 

The  following  sections  describe  the  procedure  used  in 
selecting  candidate  paint  colors,  the  field  teat,  and 
data  analysis  that  enabled  the  final  selection  of  the 
coJor(s) . 

2 . 0  COLOR  DETERMINATION 

During  the  fact  finding  trip,  soil  samples  and 
35mm  color  slides  were  taken  at  each  of  32  locales  visited. 
The  color  determination  procoss  was  a  subjective  process 
based  on  an  evaluation  of  the  soil  samples  and  slides, 
tempered  by  notes  and  mental  recall  of  the  terrain  and 
background  by  the  fact  finding  team.  Spectrophotometric 
analysis  of  the  soil  samples  was  used  to  determine  the 
proper  visual  and  near-infrared  reflectance  values. 

Hundreds  of  color  chips  in  selected  color  areas  were 
procured  from  commercial  and  government  sources  V. 

These  chips  were  matched  against  the  soil  samples  and 
the  backgrounds  as  viewed  in  the  slides.  The  evaluation 
of  soil  saiu,.  lQ  colors  took  into  account  that  soil  samples 
represented  the  color  slightly  below  the  surface.  This 
color  is  in  most  cases,  different  from  the  color  seen 
by  an  on-site  observer,  because  the  fine  sand  particles 
have  been  blown  away  by  the  desert  winds,  leaving  the 
heavier  particles  behind.  The  surface  layer,  therefore, 
appears  slightly  different  in  color  than  a  soil  sample 
which  is  drawn  from  below  the  surface.  Based  upon  the 
color  determination  process  described,  nine  camouflage 
colors  were  selected.  A  sample  quantity  of  paint  waB 
procured  under  paint  specification  TT-E-529  with  a  15-18% 
reflectance  at  6G°.  This  paint  would  provide  sufficient 
surface  hardness,  while  still  being  acceptable  (from 


the  camouflage  point  of  view)  in  teems  of  gloss.  The 
test  vehicles  consisted  of  eleven  V- 150  armored  vehicles. 

Nine  of  the  vehicles  were  painted  monotone  colors  and 
two  were  painted  in  a  two-color  pattern  specially  designed 
for  the  V-150. 

3.0  SITE  SELECTION 

At  the  request  of  SANG,  all  field  testing  was  restricted 
to  areas  around  Riyadh  and  Hofuf.  A  survey  of  the  local 
terrain  around  Riyadh  and  Hofuf  showed  that  there  were 
four  general  terrain  colors i  tan,  gray,  buff,  and  tan-red. 
Two  elevated  viewing  stations  for  the  observers  were 
selected  for  each  of  the  four  sites  of  interest.  All 
target  vehicles  (V-1501 a)  were  viewed  against  the  background 
of  the. terrain  and  not  against  the  sky.  Table  1  shows 
the  four  sites  with  the  viewing  ranges  and  site  color. 

TABLE  1 


SITE 

Tan 

Gray 

Buff 

Tan-Red 


Site  Description 

' ,  . .  i . 

NEAR  RANGE 

720  Meters 
587  Meters 
778  Meters 
738  Meters 


PAR  RANGE 


1,020  Meters 
853  Meters 
1,050  Meters 
911  Meters 


The  near  range  was  selected  to  allow  observers  close 
study  of  the  colors.  The  far  range  was  selected  to 
represent  a  quasi-realistic  combat  distance. 

4.0  TEST  METHOD 

The  method  of  testing  involved  a  ranking  technique 
using  direct  comparisons  between  two  vehicles.  This 
technique  is  more  sensitive  and  discerning  than  actual 
measurement  on  a  scale  of  values  and  helps  overcome 
the  problem  of  multidimensional  judgements,  i.e.,  inconsist¬ 
ency  of  judgements  expressed  by  the  same  observer. 

In  this  paired  method  of  ranking,  an  observer  is  presented 
with  every  possible  combination  of  two  vehicles  from 
the  set  to  be  evaluated.  The  observer  is  forced  to 
choose  between  the  two  vehicles,  and  must  decide  either 
way,  even  if  he  thinks  the  other  is  just  as  good.  The 
data  was  recorded  by  each  observer  and  was  summarized 
for  each  test  location.  It  was  analyzed  statistically 
to  determine  preferred  colors  in  the  order  of  rank, 
established  confidence  limits,  and  color  groupings  for 
each  site.  All  observations  were  a  minute  in  duration, 
and  were  performed  between  the  hours  of  1000  and  1400 
for  proper  sun  angle  and  minimum  shadows.  Each  day 


was  clear  and  hot  with  temperatures  between  120  and 
130  degrees  Fahrenheit.  Table  2  summarizes  the  number 
of  observers  for  eaoh  site  and  range.  All  observers 
were  screened!  for  20/20  visual  acuity  and  normal  color 
vision  before  selection. 

TABLE  2 


Number  of  Observers  for  Each  Site  and  Range 


SITE  RANGE 


NEAR 

FAR 

Tan 

10 

11 

Gray 

11 

11 

Buff  1 

10 

10 

Tan -Red 

10 

10 

,15.0  RESULTS 

The  mean  preference  and  95%  confidence  interval 
was  calculated  for  each  V-150  color.  These  results 
are  presented  as  descriptive  data  for  each  viewing  range 
for  each  test  sito. 


TABLE  3 


Descriptive  Data,  Tan  Terrain,  Range  1,020  Meters 


Vehicle 
Color  Number 


Mean 

Preference 


95%  Confidence  Interval  ' 
Lower  Limit  Upper  Limit 


1 

2.90 

1.86 

3.94 

2 

3.70 

2.66 

4.73 

3 

5.00 

3.96 

6.04 

4 

2.10 

1.06 

3.14 

5 

3.10 

2.06 

4.14 

6 

S .  30 

7.26 

9.34 

7 

2.30 

1.26 

3.34 

7/6 

7.70 

6.66 

8 . 74 

7/10 

4.10 

3.06 

5.14 

10 

7.40 

6.36 

8.44 

SAUDI  SAND 

6.40 

7.36 

9.44 

The  higher  the  mean  preference  number,  the  greater  the 
number  of  times  the  color  was  preferred  by  the  observers. 
That  is,  Color  Saudi  Sand  was  selected  as  being  the 
most  preferred  color  with  a  mean  preference  of  8.40. 

The  associated  confidence  interval  states  that  there 
is  95%  confidence  that  the  true  mean  preference  rests 
between  7.36  and  9.44.  The  remaining  Tables  4  through 
10  can  be  interpreted  in  the  same  manner  as  Table  3. 
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TABLE  4 


Descriptive  Data,  Tan  Terrain,  Range  720  Meters 


Vehicle 

Mean 

95%  Confidence 

Interval 

Color  Number 

Preference 

Lower  Limit 

Upper  Li: 

1 

1.45 

0.47 

2.44 

2 

4.55 

3.56 

5.53 

3 

4.64 

3.65 

5.63 

4 

4.00 

3.01 

4.99 

5 

3.64 

2.65 

4.63 

6 

8.91 

7.92 

9.90 

7 

1.91 

0.92 

2.90 

7/6 

7.36 

6.37 

8.35 

7/10 

3.75 

2.74 

4.72 

10 

6.55 

5.56 

7.53 

SAUDI  SAND 

8.27 

7.28 

9.26 

TABLE  5 


Descriptive  Data,  Gray  Terrain,  Range  853  Meters 


Vehicle 

Mean 

95%  Confidence  Interval 

Color  Number 

Preference 

Lower  Limit 

Upper  Limi 

1 

7.91 

7.07 

8.75 

2 

6.64 

5.79 

7.47 

3 

2.36 

1.52 

3.21 

4 

7.27 

6.43 

8.12 

.  5 

3.55 

2.70 

4.39 

6 

1.36 

0.52 

2.21 

7 

7.82 

6.97 

8.66 

7/6 

5.36 

4.52 

6.21 

7/10 

6.91 

6.07 

7.75 

10 

4.91 

4.07 

5.75 

SAUDI  SAND 

0.91 

0.07 

1.75 

TABLE  6 


Descriptive  Data,  Gray  Terrain,  Range  587  Meters 


Vehicle  Mean 

Color  Number  Preference 


95%  Confidence  Interval 
Lower Limit  Upper  Limit 


1  9.18 

2  8.00 

3  2.45 

4  7.45 

5  5.18 

6  1.91 

7  5.91 

7/6  3.73 

7/10  5.91 

10  4.55 

SAUDI  SAND  0.73 


8.34 

10.03 

7.16 

8.84 

1.61 

3.29 

6.61 

8.29 

4.33 

6.03 

1.07 

2.75 

5.07 

6.75 

2.88 

4.57 

5.07 

6.75 

3.70 

5.39 

0.00 

1.57 

TABLE  7 

Descriptive  Data,  Buff  Terrain,  Range  1,050  Meters 

Vehicle  Mean  95%  Confidence  Interval 

Color  Number  Preference  Lower  Limit  Upper  Limit 


TABLE  8 


Descriptive  Data,  Buff  Terrain,  Range  778  Meters 


Vehicle 
Color  Number 


Mean 

Preference 


95%  Confidence  Interval 
Lower  Limit  Upper  Li: 


1 

2 

3 

4 

5 

6 
7 

7/6 

7/10 

10 

SAUDI  SAND 


1.80 

1.10 

4.00 

3.40 

3.10 

8.50 

'3.90 

7.30 

6.70 

7.20 

8.00 


0.87 

0.17 

3.07 

2.47 

2.17 

7.57 

2.97 

6.37 

5.77 

6.27 

7.07 


2.73 

2.03 

4.93 
4.33 
4.03 
9.43 
4.83 
8.23 
7.63 
8.13 

8.93 


TABLE  9 


Descriptive  Data,  Tan-Red  Terrain,  Range  911  Meters 


Vehicle 
Color  Number 


Mean 

Preference 


95%  Confidence  Interval 
Lower  Limit  Upper  Limit 


TABLE  10 


Descriptive  Data,  Tan-Red  Terrain,  Range  738  Meters 


Vehicle 

Mean 

95%  Confidence 

Interva 1 

Color  Number 

Preference 

Lower  Limit 

Upper  Limit 

1 

1.90 

1.12 

2^68 

2 

1.40 

0.62 

2.18 

3 

5.90 

5.12 

6.68 

4 

4.00 

3.22 

4.78 

5 

4.60 

3.82 

5.38 

6 

8.40 

7.62 

9.18 

7 

1.60 

0.82 

2.38 

7/6 

7.30 

6.52 

8.08 

7/10 

3.40 

2.62 

4.18 

10 

7.40 

6.62 

8.18 

SAUDI  SAND 

9.10 

8.32 

9.88 

An  analysis  of  variance  was  performed  upon  the  data 
to  determine  the  effect  upon  the  two  ranges  as  to  color 
preference..  These  results  are  shown  in  Tablei^ll-14 . 


TABLE  11 


Analysis  of  Variance  to  Determine  the  Effect  of  Range 
and  Vehicle  Color  Upon  Color  Preference  -  Tan  Terrain 


Source 


Degrees  of  Sum  of 
Freedom  Squares 


1  0.000** 
10  1,240.914 


4.178 


578.800 


Mean 

Squares  F-Ratio 


0.000**1.00 
124.091  44.81* 


0.418  0.15 


2.769  - 


*  Significant  at  a  ■  0.01  **  Less  than  0.001 

Conclusion:  The  data  indicated  highly  significant  differences 
in  vehicle  color  preferences,  and  that  these  preferences 
are  not  affected  by  range  distances.  The  significant 
interaction  is  only  the  result  of  the  variable  vehicle 


TABLE  12 


Analysis  of  Variance  to  Determine  the  Effect  of  Range 
and  Vehicle  Color  Upon  Color  Preference  -  Gray  Terrain 


Source 

Degrees  of 
Freedom 

Sum  of 
Sauares 

Mean 

Souares  F-Ratio 

Range 

1 

0.000** 

0.000**1.00 

Vehicle  Color 
Number 

10 

1,434.181 

143.418  71.24* 

Interaction 

10 

76.909 

7.691  3.82* 

Error 

220 

442.909 

2.013 

*  Significant 

at  a  ■  0.01 

**  Less  than 

0.001 

Conclusions  The  data  indicated  highly  significant  differences 
in  vehicle  color  preferences*  and  that  these  preferences 
are  not  affected  by  range  distances.  The  'significant 
interaction  is  only  the  result  of  the  variable  vehicle 
color. 


TABLE  13 


Analysis  of 
and  Vehicle 

Variance  to  Determine  the  Effect  of  Range 
Color  Upon  Color  Preference  -  Buff  Terrain 

Source 

Degrees  of 
Freedom 

Sum  of 
Scuares 

Mean 

Squares  F-Ratio 

Range 

1 

0.000*** 

0.000**1.00 

Vehicle  Color 
Number 

10 

12,297.200 

129.720  58.64** 

Interaction 

10 

42.800 

4.280  1.92* 

Error 

198 

438.00 

2.212 

*  Significant  at  a  *  0.05  ***  Less  than  0.001 

**  Significant  at  a  ■  0.01 

Conclusions  The  data  indicates  highly  significant  differences 
in  vehicle  color  preferences,  and  that  these  preferences 
are  not  affected  by  range  distances.  The  significant 
interaction  is  only  the  result  of  the  variable  vehicle 
color . 


TABLE  14 


Analysis  of  Variance  to  Determine  the  Effect  of  Range 
and  Vehicle  Color  Upon  Color  Preference  -  Tan-Red  Terrain 


Source 

Degrees  of 
Freedom 

Sum  of 

Squares 

Mean 

Squares 

F-Ratio 

Range 

1 

0.000** 

0.000** 

1.00 

Vehicle  Color 
Number 

10 

1,563.800 

156.380 

99.82* 

Interaction 

10 

58.000 

5.800 

3.70* 

Error 

198 

310.200 

1.567 

*  Significant 

i 

at  a  ■  0.01 

**  Less  than 

0.001 

Conclusion!  The  data  indicates  highly  significant  differences 
in  vehicle  oolor  preferences,  and  that  these  preferences 
are  not  affected  by  range  distances.  The  significant 
interaction  is  only  the  result  of  the  variable  vehicle 
color* 


The  above  Tables  11-14  have  indicated  that  the 
significant  differences  found  in  this  study  are  due 
solely  to  the  variable  color.  For  this  reason,  the 
data  was  collapsed  for  the  range  variable.  This  new 
data  will  then  have  the  effect  of  doubling  the  sample 
size  within  each  cell  to  enable  more  positive  conclusions. 
Tables  15-18  present  this  data  for  each  of  the  four 
test  sites. 

TABLE  15 


Descriptive 

Data,  Collapsed 

Across  Ranges,  Tan 

Terrain 

Vehicle 

Mean 

95%  Confidence 

Interval 

Color  Number 

Preference 

Lower  Limit 

Upper  Limit 

1 

2.14 

1.42 

2.86 

2 

4.14 

3.43 

4.86 

3 

4.81 

4.09 

5.53 

4 

3.10 

2.38 

3.81 

5 

3.38 

2.66 

4.10 

6 

8.62 

7.90 

9.34 

7 

2.10 

1.38 

2.81 

7/6 

7.52 

6.81 

8.24 

7/10 

3.90 

3.19 

4.62 

10 

6.95 

6.24 

7.67 

SAUDI  SAND 

8.33 

7.62 

9.05 
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The  higher  the  mean  preference  number  the  greater  the 
number  of  times  the  color  was  preferred  by  the  observers. 
That  isr  vehicle  6  was  selected  by  the  observers  as 
being  the  most  preferred  vehicle  with  a  mean  preference 
of  8.62.  The  associated  confidence  interval  states 
that  there  is  95%  confidence  that  the  true  mean  preference 
rests  between  7.90  and  9.34.  The  remaining  Tables  16-18 
can  be  interpreted  in  the  same  manner  as  Table  15. 


TABLE  16 


Descriptive  Data,  Collapsed  Across  Ranges ,  Gray  Terrain 


Vehicle 

Mean 

95%  Confidence 

Interval 

Color  Number 

Preference 

Lower  Limit 

Upper  Limit 

1 

8.55 

7.95 

9.14 

2 

7.32 

6.72 

7.91 

3 

2.41 

1.81 

3.01 

4 

7.36 

6.77 

7.96 

5 

4.36 

3.77 

4.96 

6 

1.64 

1.04 

2.23 

7 

6.86 

6.28 

7.46 

7/6 

4.55 

3.95 

5.14 

7/10 

6.41 

5.81 

7.00 

10 

4.73 

4.13 

5.32 

SAUDI  SAND 

o.e2 

0.22 

1.41 

TABLE  17 

Descriptive  Data#  Collapsed  Across  Ranges#  Buff  Terrain 


Vehicle 

Mean 

95%  Confidence 

Interval 

Color  Number 

Preference 

Lower  Limit 

Upper  Limit 

1 

2.20 

1.54 

2.86 

2 

1.35 

0.69 

2.01 

3 

4.30 

3.64 

4.69 

4 

3.40 

2.74 

4.06 

5 

2.35 

1.69 

3.01 

6 

8.10 

7.44 

8.76 

7 

3.95 

3.29 

4.61 

7/6 

6.05 

7.39 

8.71 

7/10 

6.60 

5.94 

7.26 

10 

7.40 

6.74 

8.06 

SAUDI  SAND 

7.30 

6.64 

7.96 

NUMBER  OF  COLOR 
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VEHICLE  COLOn  NIlMDEfl  , 

Figure  A  MEAN  COLOR  PREFERENCE  WITH  95K  CONFIDENCE  INTERVALS  FOR 
EACH  VEHICLE  COLLAPSED  ACROSS  RANGES  FOR  TAN-RED  TERRAIN 


A  view  of  the  graphs  indicated  the  following  vehicle 
colors  were  preferred  for  the  listed  sites < 

o  Tan  Terrain  -  6,  7/6,  10  and  Saudi  Sand 

C  Gray  Terrain  -  1,  2,  4,  7,  and  7/10 

o  Buff  Terrain  “6,  7/6,  7/10,  10  and  Saudi  Sand 

o  Tan-Red  Terrain  -  6,  7/6,  10  and  Saudi  Sand 

Paired  comparisons  were  performed  for  the  most  preferred 
vehicle  colors  for  each  of  the  sites  to  determine  signifi 
cant  differences.  The  results  are  shown  in  Tables  19-22. 
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TABLE  19 


Paired  Comparisons  of  the  Most  Preferred  Vehicle 


Colors  on 

the 

Collapsed  Data 

-  Tan  Terrain 

Vehicle 

Color  Numbers 

F-Ratio 

Significance  Levels 

6  and  7/6 

4.S48 

a  -  0.03412 

6  and  10 

10.532 

a  "  0.00137 

6  and  Saudi  Sand 

0.310 

a  -  0.  57858 

7/6  and  10 

1.238 

a  -  0.26713 

8  and  Saudi  Sand 

2.485 

a  -  0.11647 

10  and' Saudi  Sand 

7.230 

a  -  0.00775 

Conclusion*  Vehicle 

color  6  differs 

significantly  from 

vehicle  colors  7/6 

and  10 ,  and  vehicle  color  10  differs 

significantly  from 

vehicle  color  Saudi  Sand. 

TABLE  20 

Paired  Comparisons  of  the  Most 

Preferred  Vehicle 

Colors  on  the 

Collapsed  Data 

-  Gray  Terrain 

Vehicle 

Color  Numbers 

F-Ratio 

Significance  Levels 

1  and  2 

8.230 

a  -  0.00452 

1  and  4 

7.631 

a  rn  0.00622 

1  and  7 

15.455 

a  -  0.00011 

1  and  7/10 

24.937 

a  ■  0.00000 

2  and  4 

0.011 

a  rn  1.00000 

2  and  7 

1.129 

a  -  0.28918 

2  and  7/10 

4.516 

a  -  0.03470 

4  and  7 

1.366 

a  -  0,24377 

4  and  7/10 

4.978 

a  -  0.02667 

7  and  7/10 

1.129 

a  -  0.28918 

Conclusion*  Vehicle  color  1  differs  significantly  from 
vehicle  colors  2,  4,  1,  and  7/10 f  vehicle  color  2  differs 
significantly  from  vehicle  color  7/10;  and  vehicle  color 
4  differs  significantly  from  vehicle  color  7/10. 


TABLE  21 


Paired  Comparisons  of  the  Most  Preferred  Vehicle 
Colors  on  the  Collapsed  Data  -  Buff  Terrain 

Vehicle 


Color  Numbers 

F-Ratio 

Significance  Levels 

6  and  7/6 

0.011 

a 

SSI 

1.00000 

6  and  7/10 

9.734 

a 

m 

0.00207 

6  and  10, 

2.120 

a 

m 

0.14691 

6  and  Saudi  Sand 

2.769 

a 

m 

0.09763 

7/6  and  7/10 

9.096 

a 

m 

0.00288 

7/6  and  10 

1.826 

Gt 

m 

0.17786 

7/6  and  Saudi  Sand 

2.433 

a 

m 

0.12029 

7/10  and  10 

2.769 

a 

m 

0.09763 

7/10  and  Saudi  Sand 

2.120 

a 

m 

0.14691 

10  and  Saudi  Sand 

0.043 

a 

m 

1.00000 

Conclusion!  Vehicle  color  6  differs  significantly  from 
vehicle  color  7/10,  and  vehicle  color  7/6  differs  signifi¬ 
cantly  from  vehiale  color  7/10. 


TABLE  22 

Paired  Comparisons  of  the  Most  Preferred  Vehicle 
Colors  on  the  Collapsed  Data  -<  Tan-Red  Terrain 


Vehicle 
Color  Numbers 


F-Ratio 


Significance  Levels 


6  and  7/6 

3.590 

a 

■ 

0.05957 

6  and  10 

10.787 

a 

■1 

0.00121 

6  and  Saudi  Sand 

1.596 

a 

■ 

0.20799 

7/6  and  10 

1.931 

a 

■1 

0.16623 

7/6  and  Saudi  Sand 

9.973 

a 

m 

0.00184 

10  and  Saudi  Sand 

20.681 

a 

m 

0.0001 

Conclusion!  Vehicle  color  6  differs  significantly  from 
vehicle  color  10;  vehicle  color  7/6  differs  significantly 
from  vehicle  color  Saudi  Sand;  and  vehicle  color  10 
differs  significantly  from  vehicle  color  Saudi  Sand. 

The  analysis  of  the  data  indicated  that  the  color 
preferences  fell  into  two  distinct  groupings.  The  colors 
6,  7/6,  10,  and  Saudi  Sand  were  most  preferred  for  the 
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tan,  buff,  and  tan-red  terrains  (Figures  1,  3,  and  4). 
The  colors  1,  2,  4,  7,  and  9  were  the  most  preferred 
for  the  gray  colored  terrain  (Figure  2).  A  review  of 
Tables  19,  21,  and  22  show  that  a  ranking  of  the  best 
colors  for  the  tan,  buff,  and  tan-red  terrains  would 


o  Colors  6  and  Saudi  Sand  -  Best 

o  Colors  7/6  and  10  -  Next  Best 

A  review  of  Table  20  for  the  gray  color  terrain  shows 
that  the  ranking  of  the  preferred  colors  would  be  as 
follows  t 

o.  Color  1  -  Best 
o  Colors  2  and  4  -  Average 
o  Colors  7  and  7/10  -  Worst 
6.0  DISCUSSION 

Observations  by  the  test  team  made  during  this 
trip  and  during  the  fact  finding  trip  in  September  1980, 
indicated  a  predominance  of  the  tan,  buff,  and  tan-red 
backgrounds  around  the  Riyadh  and  Hofuf  vicinities. 

The  gray  test  area  was  the  exception.  Although  the 
Belvoir  Research  and  Development  Center  field  test  personnel 
did  not  have  the  opportunity  to  determine  the  physical 
extent  of  the  gray  colored  terrain,  this  color  was  not 
prevalent  at  Hofuf  or  anywhere  else  in  the  Riyadh  area. 

The  overall  color  selections  should  be  based  on 
what  is  considered  to  be  the  predominant  terrain  background 
colors.  The  data  analysis  indicated  that  the  best  color 
selection  for  the  predominant  terrain  background  (i.e., 
tan,  buff,  tan-red)  are  the  colors  6  and  Saudi  Saj^d. 

The  data  analysis  further  indicated  that  there  is  no 
significant  statistical  difference  between  these  colors. 
Color  6  tends  to  blend  in  slightly  better  with  the  tan 
and  buff  backgrounds,  whereas  color  Saudi  Sand  is  slightly 
ahead  in  the  tan-red  background.  A  forced  selection 
between  the  two  colors  would  probably  favor  color  6 
based  upon  the  test  results  in  the  tan  and  buff  sites, 
and  the  overall  preponderance  of  tan  and  buff  backgrounds 
in  the  Riyadh  and  Hofuf  areas. 

If  for  some  reason  the  gray  colored  background 
cannot  be  discounted,  then  a  color  should  be  selected 
which  is  a  compromise  for  all  sites.  Table  19  indicates 
that  colors  5,  7/6,  and  10  are  in  the  second  grouping 
of  best  colors.  Colors  7/6  and  10  are  also  in  the  next 


best  grouping  of  colors  for  the  tan,  buff,  and  tan-red 
litas  as  indicated  in  the  previous  section.  Data  analysis 
also  Indicates  that  color  7/6  does  not  differ  significantly 
from  dolor  10.  Color#  7/6  and  10  could  therefore  serve 
as  overall  compromise  colors  across  all  sites.  Although 
color  7/6  has  a  slight  edge  in  preference  across  all 
sites,  the  preferred  color  is  color  10.  This  is  based 
upon  the  camouflage  guideline  of  selecting  the  lighter 
color  when  confronted  with  a  choice  for  a  desert  background. 
The  lighter  color  tends  to  counteract  some  of  the  natural 
on-vehicle  shadjows  which  contribute  to  giving  the  vehicle 

i  a  dark  appearance.  •  • 

7.0  SUMMARY 

r  •  »■  i,  • 

A  total  of  eleven  V-150  vehicles  painted  in  camouflage 
colors  wars  viewed  by  a  minimum  of  tan  ground  observers 
at  tan,  buff,  gray,  and  tan-red  colored  backgrounds 
in  Saudi  Arabia.  Each  site  had  a  near  and  far  range. 

Every  possible  color  pair  combination  was  viewed  at 
each  site  and  range.  In  each  case,  the  observer  was 
forced  to  make  a  choice  ae  to  which  color  blended  best 
with  the  background. 

Analysis  of  the  data  indicated  two  preferred  colors 
for  the  predominant  background  terrains  found  in  the 
Riyadh  and  Hofuf  areas,  namely  oolor  6  and  color  Saudi 
Sand.  The  data  analysis  also  Indicated  that  there  is 
no  significant  statistical  difference  between  these 
colors.  A  subjective  forced  choice  between  these  colors 
favors  the  selection  of  oolor  6. 
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TETRACHORIC  CORRELATION  FOR  MESOSCALE 
AREAL  PERSISTENCE  OF  CLOUD  CEILINGS 


.  Oskar  M.  Essenwanger 
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ABSTRACT ;  The  correlation  coefficient  of  attributes  (tetrachoric  correlation 
for  2k 2  contingency  table)  In  a  useful  tool  In  the  study  of  areal  persistence.  . 
This  correlation  permits  us  to  evaluate  persistence  for  subgroups  (classes  of 
attributes)  while  the  ordinary  linear  correlation  coefficient  Is  a  summary 
of  the  total  data  set.  From  the  examination  of  the  tetrachoric  correlation 
It  Is  learned  that  little  persistence  Is  found  for  low  visibility  and  low 
cloud  celling  conditions  In  a  selected  mesoscale  area.  Clouds  with  a  celling 
over  8000  ft,  however,  display  an  apparent  wider  coverage  of  this  area  and 
show  persistence.  Thus  the  vertical  structure  of  persistence  Is  confirmed 
In  quantitative  numbers  by  the  tetrachoric  correlation  coefficient. 

1.  INTRODUCTION;  The  tetrachoric  correlation  coefficient  of  a  2X2  contingency 
table  Is  not  used  very  often  In  statistical  analysis  because  many  statisticians 
have  pointed  out  Its  limitations,  especially  the  problem  of  determining 
Its  statistical  significance.  In  turn,  the  ordinary  linear  correlation 
coefficient  may  be  of  limited  value  if  applied  to  non-Gausslan  distributions 
such  as  cloud  amounts  (sky  cover)  and  ceiling.  Furthermore,  areal  persistence 
as  judged  by  the  ordinary  linear  correlation  coefficient  is  not  suitable  to 
disclose  vertical  stratification  of  areal  persistence.  In  this  particular 
case  the  tetrachoric  correlation  Is  not  only  a  useful  tool  to  measure  areal 
persistence  in  the  mesoscale,  but  can  reveal  the  vertical  structure  of  areal 
persistence.  We  learn  from  cloud  data  of  a  mesoscale  area  in  Central  Europe 
that  fog  and  low  ceilings  follow  a  Joint  probability  which  Is  virtually 
equivalent  to  randomness.  However,  ceilings  above  4000  feet  exhibit  non¬ 
random  behavior,  and  cloud  ceilings  extend  over  a  much  larger  area  with 
consistency.  This  Interpretation  Is  obtained  in  quantitative  form  from  the 
use  of  the  tetrachoric  correlation  coefficient. 


*  *  J  *  •  *'  *  -4  '  *  * 


2.  THEORETICAL  BACKGROUND,  Let  the  four  fields  of  a  2X2  contingency  table 
be  caVlecl  aT  b,  c,  d  (Fig  1).  Then  the  marginal  distribution  can  be  written: 


a  b 

Rl 

c  d 

*2 

r-j  ■  a  +  b 

(la) 

Si  S2 

Tot 

R£  ■  c  +  d 

(1b) 

Fig  1  Contingency  Table 

S<j  ■  a  +  c 

(1c) 

Sg  ■  b  +  d  (Id) 


Let  us  assume  that  the  field  attributes  are  given  as  fractions  of  the  total  H, 
namely: 


and 


a  +  b  +  c  +  d-1.0  (2a) 

R-j  +  Rg  ■  S-|  +  Sg  ■  1.0  (2b) 


The  tetrachorlc  correlation  coefficient  as  defined  by  Kendall  and  Stuart  (1958) 


Is: 

O0 

w  - 

d  ■  Spt  Ti(hlxJ (k) 

(3«) 

where 

I  xr  (x)  Is  the  tetrachorlc  functions 

Tr(x)  ■  Hr-i  (x)  <»(x)/(r!  )fr 

i 

(3b) 

and 

<l>(x)  Is  the  Gaussian  integral  value  Integrated  from  -ooto 

Hr  (x)  represents  the  Tchebycheff  -  Hermlte  polynomials,  and  h  »  $2»k  ■  Rg. 


It  Is  evident  thatp  cannot  readily  be  calculated  and  Is  usually  determined  by 
Iterations  and/or  the  aid  of  tables.  Approximations  have  been  suggested  such 
as: 

rj  •*  [(  »72)(/a3  -  /5c)/(  /a3  +  /be)]  (4) 


The  form  which  Is  used  here  Is  based  on  the  calculation  and  is  sometimes  called 
the  phi-coefficient.  The  (linear)  correlation  coefficient  of  attributes  In  a 
contingency  table  Is: 

ra  *  [(  x2  /  N)/(m-1 ) (5) 
and  for  a  2X2  contingency  table  wUh  m  ■  2,  we  find: 
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X^/N  *  (ad  -  be)2  /  (R-)  R2  S-|  S2)  (6a) 

although  Yates  has  suggested  a  corrected  form  (adjusted  to  eqn  2a,  b): 
x2/N  ■  (  |ad  -  be)  -  1/2N)2  (6b) 


which  must  be  used  for  Small  N.  In  our  case  N>200  and  the  correction  can  be 
neglected.  Hence: 


r\  *  (ad  -  be)2/  (Ri  R2  Si  S2) 


which  can  be  transformed  Into: 


-  r{  ■  (a  -  RiS<j)2  /  (^  R2  S2)  (8) 

This  has  also  been  called  the  tetrachorlc  correlation  by  various  authors. 


The  reader  will  notice  that  rt  *  0  for  a  ■  Si  Ri  and  rt  ■  1.0  for  Si  -  Ri  »  a. 
The  first  case  a  ■  Si  Ri  represents  the  joint  probability  of  two  events 
occurring  at  random.  The  second  case  Implies  a  perfect  match  of  the  two 
events,  or  complete  association.  The  size  of  rt  for  a  2X2  contingency  table  of 
the  joint  probability  of  cloud  cover  expresses  the  degree  of  persistence. 
Caution  must  be  exercised,  however. 


Similar  top  or  rj  the  establishment  of  the  statlst.lgal  significance  for 
Is  not  simple,  this  Is  supported  by  the  fact  that  for  S2  t  Ri  the  correlation 
rt  has  a  maximum  value  rmax<1.0  (see  Table  1).  Thus  the  Interpretation 
of  persistence  from  the  use  of  rt  is  not  trivial,  In  order  to  evaluate  the 
effect  of  the  asymmetry  Ri  t  Si  we  may  define  a  ratio: 


it  "  rt/rmax  (9a) 

Assume  Ri  -  Si ,  the  field  a-Ri,  then  amax  ■  Ri;  for  R1-S1  we  have  amax 

r2max  "  R1  Vr2  S1  (9b) 

or  nj  ■  ( a  -  Ri  S^2  /[Rf  (1  -  S^2]  (9c) 

For  an  observed  a0bs  we  finalize: 

nt  ■  («obs  -  R1  S1  )/R1  S2!  ®obs  "R1  W 


Table  1  displays  the  maximum  rt  for  Ri  and  S^  Since  the  values  of  rt  are 
symmetric  around  the  diagonal  only  one  part  is  provided.  Evidently  the  lower 


the  ratio  Ri/S2  the  lower  Is  rmax.  However,  rmax  Is  not  linearly  related  to 
Ri /Si  (see  eqn.  9b). 


The  ratio  nt  provides  an  evaluation  whether  rt  has  rendered  Its  maximum 


me  ratiu  nt  pruviues  an  evaluation  wneiner  r+  nas  renaerea 
value,  and  can  be  considered  an  adjusted  measure  of  persistence. 


The  utilization  of  r^  has  some  advantages  compared  with  the  application 
of  the  ordinary  linear  correlation  coefficient.  The  calculation  of  rt  Is  based 
on  a  contingency  table.  Thus  the  correlation  (and  Implicitly  the  persistence) 
can  be  determined  or  specific  groups  of  cloud  conditions  by  establishing 
the  contingency  taules  for  these  groups  and  comparing  r+  for  them.  In  our  case 
these  groups  are  arranged  by  classes  of  the  cumulative  frequency  of  occurrence 
of  cloud  celling  by  altitude,  although  the  contingency  tables  needed  for 
comparison  can  be  established  for  any  grouping  of  attributes.  The  grouping 
of  cloud  cover  by  celling  height  permits  us  to  study  the  variation  of  the 
persistence  as  a  function  of  celling  altitude,  Information  which  would  not 
readily  be  available  from  the  ordinary  linear  correlation  coefficient  which 
Is  calculated  for  the  entire  data  set. 

For  the  referenced  data  sample  the  correlation  coefficient  can  be  considered 
as  significantly  different  from  zero  at  the  95*  level  of  significance  for  r- 0.055 
(one-sided)  or  r*  0.067  (two-sided).  The  respective  significance  for  the  three 
sigma  level  would  be  rfcO.IO  and  0.11,  respectively.  These  values  will  be 
referred  to  In  the  subsequent  text. 


3.  GROUPS  OF  CLOUDS.  As  previously  outlined  the  question  is  whether  persistence 
Is  the  same  Irrespective  of  celling  height  and  conditions  designated  as 
''adverse  weather".  Celling  In  this  connection  Is  defined  as  cloud  cover  of 
-0.5  of  the  sky.  The  following  cases  were  of  Interest. 

0  visibility  -  1  km  (fog) 

0  celling  -  500  ft 

0  celling  -  500  ft  and/or  visibility  -  2  ml 

0  ceiling  S  800  ft 

0  celling  ^  800  ft  and/or  visibility  •  3  ml 

0  veiling  -  2000  ft  and/or  visibility  -  4  ml 

e  celling  -  4500  ft  and/or  visibility  -  4  mi 

0  ceiling  "  8000  ft  and/or  visibility  ■  5  mi 


The  frequency  of  occurrence  and  the  joint  (empirical)  probabilities  have  been 
compiled  In  reports  by  Essenwanger  and  Levitt  (1984)  and  Levitt  and 
Tssenwanger  (1984).  Table  2  is  an  example  of  these  joint  probabilities.  The 
joint  (empirical)  probability  of  "adverse  weather"  is  reasonable  as  displayed 
in  Table  2  for  the  selected  groups.  The  question  remains  whether  this  Is  the 
lowest  possible  joint  probability.  It  would  be  If  random  association  can  be 
assumed  but  It  would  not  be  If  the  area  persistence  of  adverse  weather  is  high 
between  the  two  stations.  This  will  be  Investigated  now. 


4.  TWO  STATION  AREAL  PERSISTENCE.  As  a  first  example  the  condition  for  fog 
(visibility  S  1  km)  Is  examined.  Table  3a  exhibits  the  occurrence  of  fog  at 
four  stations  (central  Europe)  In  fall.  Table  3b  lists  the  joint  probabilities 
for  the  six  possible  combinations  of  pairs  arranged  by  distance.  This  table 
exhibits  a  decrease  of  the  joint  occurrence  with  distance  and  a  slight  Increase 
of  the  probabilities  In  the  last  two  columns. 


. 


•  v' 


* 
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How  much  is  this  increase  caused  by  a  higher  persistence?  This  question 
can  be  answered  from  Table  3c  which  discloses  rt  for  the  six  station  pairs. 

The  first  apparent  difference  between  Tables  3b  and  c  is  the  fact  that  the 
joint  probability  displays  a  different  diurnal  trend.  While  the  highest  joint 
probability  Is  found  at  7 ^  in  the  morning  the  highest  persistence  Is  found 
between  19**  to  01 11  except  for  the  last  two  columns  where  highest  persistence  and 
peak  of  joint  probability  coincide.  These  variations  as  depicted  are  daused 
by  the  differences  of  terrain  conditions  within  the  mesoscale  area.  These 
differences  are  again  evident  in  the  comparison  of  the  station  pairs  as  function 
of  the  distance. 

It  can  be  noticed  that  the  correlation  (and  persistence)  shows  a  tendency 
to  decrease  with  distance  although  the  last  two  columns  disclose  again  a  slight 
Increase  In  rt  in  comparison  with  the  middle  range  of  70  miles.  This  small 
Increase  may  be  a  terrain  effect.  Apparently  fog  conditions  In  the  Rhine  Valley 
show  a  small  tendency  to  occur  also  In  the  Saar  Valley  while  Hahn  which  Is 
located  In  a  mountain  area  does  not  correlate  as  well  with  Frankfurt  and 
Heidelberg.  The  small  value  of  rt  Indicates  very  little  persistence.  It  may 
surprise  that  even  Heidelberg  and  Frankfurt,  both  In  the  Rhine  Valley,  display 
only  a  moderate  persistence,  l.e.  a  maximum  association  of  38%  (from  rt  *  0.62), 
and  this  only  at  22h  (local  time). 

Table  4  is  selected  to  show  the  variation  of  persistence  with  altitude  and 
distance.  Only  two  levels  of  ceiling  are  shown,  under  800  and  8000  ft.  A 
comparison  of  rt  for  the  top  and  the  lower  part  of  Table  4  reveals  that  for 
all  but  one  column  the  correlations  are  generally  higher  If  the  celling  height 
Is  higher.  This  result  Is  not  changed  by  the  ratio  hr  (eqn.  9d). 

Again,  a  marked  decrease  Is  evident  with  distance  Implying  that  persistence 
decreases  with  distance.  Although  this  result  is  expected  It  Is  a  quantitative 
formulation  exhibited  in  Table  4.  The  high  correlation  for  celling  S800  ft 
for  the  station  pair  Heidelberg  -  Frankfurt  contrasting  the  lower  value  for 
celling  under  8000. ft  (column  three)  may  be  somewhat  unexpected.  It  has  some 
explanation  In  the  terrain  conditions  within  the  mesoscale  area,  however. 

It  was  pointed  out  In  the  discussion  of  Table  3c  that  Heidelberg  and 
Frankfurt,  both  In  the  Rhine  Valley,  show  a  high  correlation  (and  persistence) 
for  fog.  This  high  persistence  Is  extended  to  Include  the  conditions  celling 
■  800  ft.  However,  If  celling  * 8000  ft  Is  considered  both  stations  show  a 
more  Independent  pattern.  This  conclusion  could  not  be  deduced  from  Inspection 
of  Table  2  where  the  joint  probabilities  are  of  the  same  magnitude  and  little 
distance  dependency  Is  shown. 

5.  THREE  STATION  AREAL  PERSISTENCE.  Examples  of  three  station  areal  correlations 
are  presented  In  Tables  5  and  6.  Tn  the  case  of  three  stations  we  have  the 
option  of  utilizing  a  2X2  or  3X3  contingency  table.  For  the  2X2  contingency 
table  we  start  with  the  joint  probability  for  one  station  pair  as  Ri  or  S-| 

(see  eqn.'s  1)  and  "A*  denotes  the  joint  probability  of  the  three  stations.  This 
scheme  Is  similar  to  the  algorithm  which  was  discussed  before.  An  example  Is 
given  In  Table  5. 
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We  learn  from  Table  5  that  the  correlation  increases  with  Increasing 
ceiling  altitude.  This  implies  that  cloud  layers  in  higher  altitudes  show 
a  larger  areal  persistence  or  area  coverage.  This  fact  can  be  confirmed 
from  Inspection  of  dally  weather  maps  and  study  of  their  frontal  systems. 

This  persistence  depends,  however,  on  the  size  of  the  mesoscale  area.  In 
Table  5  the  diurnal  variation  was  eliminated  by  averaging  although  from  a  point 
of  theoretical  statistic  Fisher's  z-functlon  should  have  been  used.  The 
calculation  did  not  alter  the  exhibited  relative  magnitude  within  Table  5  and 
Is  therefore  omitted. 

It  is  noted  that  the  condition  of  ceiling  5  2000  ft  without  inclusion  of 
visibility  displays  a  slightly  smaller  correlation  than  with  inclusion  of  the 
visibility  except  for  the  last  column.  This  Increase  of  the  persistence  Is 
reasonable  because  good  visibility  appears  to  prevail  over  a  larger  area  than  poor 
visibility.  The  latter  Is  a  predominant  effect  of  local  terrain  which  is 
quantitatively  confirmed  by  the  correlation. 

Table  6  exhibits  an  additional  feature.  While  part  A  of  the  table 
provides  the  tetrachoric  correlation  part  B  shows  the  correlation  based  on 
eqn.  5  and  the  3X3  contingency  table.  In  the  latter  case  the  total  area  coverage 
is  judged.  Part  A  permits  us  to  study  the  persistence  In  segments  of  the 
mesoscale  area.  The  three  station  correlation  in  part  B  Is  only  one  single 
measure  for  the  total  area  cover.  Both  measures  reflect  the  Increase  in  areal 
persistence  from  lower  to  higher  celling  threshold.  Furthermore,  Table  6A 
also  discloses  some  seasonal  variations.  For  low  celling  (under  800  ft) 
little  areal  persistence  exists  In  summer  while  the  winter  season  shows 
relatively  strong  persistence.  Again,  this  Is  expected.  During  the  summer 
months  low  celling  Is  limited  to  small  areas  associated  with  terrain  conditions. 

In  Central  Europe  fog  and  low  celling  prevail  often  over  even  larger  areas 
In  wintertime  than  the  mesoscale  area  under  study  here.  The  tendency  of  lower 
persistence  in  summer  for  low  celling  Is  also  confirmed  In  Table  6B. 

5.  FOUR  STATION  AREAL  PERSISTENCE.  Again,  several  choices  are  available  how 
to  produce  a  2X2  contingency  table  for  four  stations  and  examine  the  areal 
persistence.  The  first  example  Is  given  in  Table  7.  The  Joint  probability  of 
adverse  weather  Is  compared  with  R-|  and  S-)  being  the  joint  probability  of  two 
stations.  This  permits  us  a  study  of  areal  persistence  in  the  zonal  and 
meridional  direction.  The  correlation  and  ratio  of  correlation  to  the  maximum 
possible  correlation  Is  exhibited  in  Table  7  for  four  seasons. 

We  learn  that  again  low  visibility  and  celling  show  little  areal  persistence 
In  the  zonal  direction  while  In  the  meridional  direction  areal  persistence  Is 
low  In  spring.  Areal  persistence  Improves  with  higher  altitude  of  the  celling. 
Apparently  the  persistence  Is  about  equal  In  the  zonal  and  meridional  direction 
for  higher  celling.  The  numerical  values  increase  for  the  ratio  nt  but  the 
general  trend  of  the  areal  persistence  remains  the  same.  The  higher  areal 
persistence  In  the  meridional  than  the  zonal  direction  for  low  visibility  and 
celling  altitude  may  be  a  consequence  of  frontal  systems  predominantly  moving 
In  the  meridional  direction  In  this  particular  mesoscale  area. 


Table  8  provides  a  summary  of  the  diurnal  trend  for  selected  groups  of 
adverse  weather  In  fall.  This  time  the  correlation  Is  calculated  from  R<| 
being  the  joint  probability  of  three  stations  and  Si  the  probability  of  the 
fourth  station.  Four  conditions  (one  for  each  station  as  S-|)  can  be  construed. 

The  correlations  for  these  four  conditions  have  been  arranged  for  specified 
hours.  It  Is  evident  that  a  definite  diurnal  trend  exists.  The  minimum 
persistence  Is  underlined  In  Table  8.  The  diurnal  difference  of  the  correlation 
appears  to  be  reduced  with  Increasing  celling  altitude  but  the  minimum  persistence 
does  not  display  a  uniform  pattern. 

7.  CONCLUSION.  It  has  been  Illustrated  that  the  correlation  coefficient  of 
attributes  for  a  contingency  table  (for  a  2X2  contingency  table  also  called 
tetrachorlc  correlation)  can  be  useful  In  the  study  of  areal  persistence  of 
cloud  and  visibility  conditions.  Since  the  fields  of  the  contingency  table 
can  reflect  the  probability  of  occurrence  for  specified  classes  It  Is  possible 
to  study  the  vertical  structure  of  the  areal  persistence  while  the  ordinary 
linear  correlation  coefficient  can  only  express  the  persistence  for  the  total 
set  of  data. 

It  has  been  shown  that  persistence  for  a  mesoscale  area  depends  on  celling 
altitude.  It  was  deduced  that  low  visibility  ($1  km)  and  low  celling  (  -1000  ft) 
Is  predominately  terrain  dependent  with  only  a  small  areal  persistence.  However, 
clouds  with  celling -8000  ft  tend  to  cover  more  than  the  mesoscale  area  under 
Investigation.  Thus  they  depict  areal  persistence.  As  Table  7  exhibits  there  Is 
little  seasonal  fluctuation  for  celling  58000  ft. 

The  data  on  cloud  cover  over  a  mesoscale  area  In  Central  Europe  produce 
asymmetric  2X2  contingency  tables.  In  this  case  the  maximum  possible  correlation 
Is  smaller  than  unity,  A  correction  can  be  mad,e  to  evaluate  this  bias  by 
calculating  a  ratio  nt  of  the  correlation  coefficient  to  the  maximum  correlation. 

As  It  proved  the  result  concerning  the  areal  persistence  does  not  change 
significantly  by  examination  of  this  ratio. 
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Saarbruecken,  HA  =  Hahn,  HEI  =  Heidelberg,  FR  =  Frankfurt 


TABLE  3A  VISIBILITY  <  1  KM  (PROBABILITIES) 

FALL 

UQUft _ HUB. _ EBAHK - M - HAUL 


1 

S.3X 

MX 

8.2X 

10.7X 

A 

6,2 

7,9 

12.0 

13.8 

7 

9.2 

10.0 

16.9 

19.3 

10 

5,9 

6,9 

12.4 

13.7 

13 

2,6 

3,1 

4.3: 

6.6 

16 

2,6 

2.4  ‘ 

2.2 

5,6 

19 

2,9 

2.5 

3.8 

6.5 

22 

4,2 

3,2 

4.1 

8,1 

HEIDELBERG 

FRANKFURT 

•AAR1RUICKIN 

HAHN 


TABLE  31  JOINT  FROBABIUTICS  OF  VISIBILITY *1KN  FOR  STATIONS  OF  TABLE  SA 


VISIBILITY 
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TABLE  4  TWO-STATION  CORRELATIONS  (FALL) 


C  <  800  FT 

Tetraqhoric  Correlation 


MILES: 

40 

50 

70 

70 

80 

90 

HOUR 

SA-HA 

HEI-FR 

HEI-HA 

FR-HA 

HEI-SA 

SA-FR 

1 

.44 

.58 

.17 

.22 

.21 

.30 

7 

.37 

.46 

.09 

.21 

.15 

.19 

13 

.39 

.37 

.11 

.25 

.19 

.28 

19 

.47 

.46 

.20 

.35 

.19 

.24 

RATIO 

1 

.53 

.58 

.32 

.41 

.34 

.48 

7 

.46 

.53 

.18 

.42 

.25 

.27 

13 

.44 

.48 

.24 

.53 

.37  ' 

.41 

19 

.56 

.55 

.39 

.70 

.32 

.33 

C 

i  8000  FT 

■■ 

Tetrachorlc  Correlation 

MILES: 

40 

50 

70 

70 

80 

90 

HOUR 

SA-HA 

HEI-FR 

HEI-HA 

FR-HA 

HEI-SA 

SA-FR 

1 

.66 

.23 

.31 

.34 

.29 

.31 

7 

.60 

.20 

.31 

.29 

.24 

.23 

13 

.66 

.23 

.09 

.15 

.15 

.19 

19 

.59 

.22 

.37 

.32 

.19 

.25 

RATIO 

1 

.73 

.25 

.36 

.42 

.30 

.31 

7 

.64 

.22 

.33 

.34 

.26 

.24 

13 

.72 

.24 

.10 

.16 

.15 

.20 

19 

.62 

.22 

.42 

.37 

.20 

.26 

C  -  CLOUD  CEILING 
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TABLE  6  THREE-STATION  TOTAL 


TABLE  7 


FOUR  STATION  PERSISTENCE 
A)  CORRELATION 


E- 

W 

N- 

S 

SP 

SU 

FA 

WI 

SP 

SU 

FA 

WI 

V£1  KM 

.06 

(.05) 

.13 

.12 

.09 

— 

.32 

.25 

800  FT 

.09 

.10 

.14 

.15 

.12 

.31 

.34 

CM 

• 

C  5  2000  FT 

.25 

.21 

.25 

.31 

.32 

.27 

.43 

.43 

C  £4500  FT 

.43 

.36 

.42 

.42 

.44 

.36 

.48 

.45 

C  58000  FT 

.46 

.40 

.50 

.47 

.47 

.41 

.50 

.48 

C  5500  FT/V52  MI 

.16 

.17 

.24 

.24 

.17 

.23 

.41 

.32 

C 5 2000  FT/ V54  MI 

.36 

.33 

.35 

.34 

.43 

.36 

.46 

.43 

C 5 8000  FT/V55  MI 

.45 

.41 

.43 

.40 

.47 

.44 

.45 

.39 

B)  RATIO 

E-W 

N- 

■S 

SP 

SU 

FA 

WI 

SP 

SU 

FA 

WI 

V  1  KM 

.25 

(.30) 

.16 

.27 

.10 

-- 

.39 

.33 

C  800  FT 

.40 

.30 

.28 

.36 

.16 

.51 

.41 

.35 

C  2000  FT 

.67 

.61 

.45 

.56 

.35 

.28 

.54 

.49 

C  4500  FT 

.63 

.62 

.56 

.52 

.46 

.39 

.52 

.50 

C  8000  FT 

.55 

.47 

.52 

.50 

.48 

.44 

.52 

.49 

C  —  500  FT/V52  MI 

.39 

.47 

.33 

.43 

.21 

.30 

.45 

.38 

C  5  2000  FJ/m  MI 

.64 

.57 

.48 

.57 

.50 

.40 

.49 

.49 

C  58000  FT/V55  MI 

.55 

.49 

.48 

.50 

.50 

.49 

.48 

.42 

E-W  «  Hahn/Saarbruecken  -  Frankfurt/Heidelberg 
N-S  »  Frankfurt/Hahn  -  Saarbruecken/Heldelberg 


TABLE  8 


FOUR  STATIONS#  FALL 
Area  Persistence  (Kean  Correlation) 


1 

.26 

.28 

.50 

.41 

A 

.22 

.23 

.45 

.35 

7 

.17 

.16 

.46 

J2 

10 

#11 

.10 

.fc 

.34 

13 

.14 

JZ 

.38 

16 

.17 

.17 

.41 

.43 

19 

.20 

.20 

.45 

.44 

22 

.26 

.23 

.50 

.42 

(CORRELATIONS  OE  POUR  COMBINATIONS  "THREE “STATION 
VERSUS  ONe  STATION"  AVERAOED) 
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NQNPARAMETRIC  MEDIAN  ESTIMATION 
(With  Application  to  Number  of  Simulation  Replications  Needed) 

J.  R.  Knaub,  Jr. 

U.S.  Army  Logistics  Center 


When  the  variance  of  results  from  replications  of  a  simulation  Is  small,  It 
1$  Intuitively  apparent  that  a  small  sample  size  (small  number  of  replications) 
will  be  sufficient,  tf  four  replications  are  made,  for  Instance,  and  the 
difference  between  the  smallest  and  largest  of  the  four  observations  Is  of 
little  or  no  practical  Importance,  this  sample  size  may  be  sufficient. 


Often,  ten  observations  can  be  thought  of  as  a  minimum  sample  size, 
because,  using  distribution-free  tolerance  limits,  there  Is  a  75.6%  confidence 
that  75.0%  or  more  of  the  population  lies  between  the  extremes  of  the  ten 
values  observed.  With  four  observations,  this  confidence  is  only  26.2%. 

If,  as  In  the  example  above,  the  difference  between  the  smallest  (o^)  and 

the  largest  (o^)  of  four  observations  Is  of  little  or  no  practical  Importance, 

It  Is  of  Interest  to  find  the  probability  that  the  median  of  the  population 
lies  between  the  extremes  of  the  four  observations1.  This  probability  Is 

1-2  (H)*  B  0.875.  If  the  central  limit  theorem  Is  applicable'*  then  the  median 
Is  approximately  equivalent  to  the  mean.  Thus,  the  mean  of  the  sample,  In  this 
case,  Is  very  likely  to  be  very  close  to  the  mean  of  the  population. 

Suppose  that  seven  replications  are  made  Instead  of  four.  There  Is  a 
probability  of  0.875  that  the  population  median  (M)  will  fall  between  the 
second  smallest  (o^)  and  second  largest  (og)  observations,  which  Is  the  same 

probability  that  M  falls  between  o^  and  o^  out  of  four  observations.  The 

difference  Is  that  Og  -  Og  In  the  case  of  seven  observations,  Is  probably 

smaller  than  -  Oj  for  the  case  of  four  observations.  See  Table  I  for 

details.  Column  1  In  this  table  Is  the  sample  size.  Column  2  Is  the  probabil¬ 
ity  that  M  will  be  located  between  the  extremes  of  the  sample  of  size  n. 

Columns  3  and  4  provide  the  probabilities  that  the  10%  and  5%  tails,  respec¬ 
tively,  of  the  population  will  be  drawn  from  at  least  once  In  n  attempts. 
Columns  5,  6,  and  7  provide  similar  Information,  but  for  the  case  where  Oj  and 

oR  are  Ignored.  Columns  8  and  9  are  distribution-free  tolerance  limit  results. 
They  provide  the  probabilities  that  50%  or  more,  or  75%  or  more  of  the  popula¬ 
tion  Is  to  be  found  between  o.  and  o„,  the  first  and  n^  ordered  observations 

l  n 

(ordered  from  smallest  to  largest). 

Although  a  fixed  sample  size  approach  could  be  taken  using  the  tables 
provided  here,  and  some  prior  knowledge  as  to  what  variance  will  be  experienced 
In  most  cases,  a  sequential  approach  would  appear  to  often  be  more  appropriate. 
Consider  the  following  equation: 


*  rne  central  limit  theorem  is  applicants  If  each  observation  Is  actually 
the  mean  of  a  number  of  observations  • 
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P,0J  6  M  ‘  °n+1-j> 


probability  that  the  median  of  the 

t  K 

population  lies  between  the  jtn 
smallest  and  jth  largest  of  n 
observations i  where  Oj  Is  the  j 
ordered  observation. 


■  1 


If  one  first  determines  the  minimum  acceptable  value  for  P[°j  5  M  S  °n+l-d’ 

and  the  maximum  acceptable  value  for  °n+l-J  "  °j  •  then  one 

may  Increase  n  and  J  until  these  conditions  are  satisfied. 

Table  II  contains  values  for  PIOj  ■  M  ■  °n+l-J^  for  selected 

values  of  n  and  j.  The  rows  are  labelled  by  sample  size  (n)  and  the  columns 
by  the  number  of  ordered  observations  (J  •  1)  eliminated  .from  both  the  low 
and  high  ends  of  the  set  of  ordered  observations.  (Therefore  2J-2  observa¬ 
tions  are  eliminated.) 

To  use  Table  II  In  a  sequential  manner,  first  determine  the  value  of 
°n+l*j  "  °j  t0  ba  USQa»  or  Perhaps  <°n+i-j  "  °j)/>f 

and  the  value  of  pt°j  ■  ^  “  °n+l-j^ .  will  be  read  from  the  table  to  deter¬ 
mine  the  minimum  sample  size  which  could  possibly  satisfy  the  former  criterion. 
If  conditions  arc  not  satisfied,  look  In  tho  table  to  see  how  many  more  obser¬ 
vations  must  be  takon  so  that  J  may  be  Incremented  by  one,  and  the  criterion 

for  p(°j  ■  M  ■  met.  After  those  additional  observations 

are  taken,  the  other  criterion  ^e,S*»°n+i-j  *  °j^  can  be  checked  for  the 


new  case,  This  process  Is  repeated  until  both  criteria  are  met. 

Example; 

Let  the  criteria  be  that  P t o j  ■  M  ■  on+^.j]  must  be  greater  than 

or  equal  to  0.90,  and  that  °n+l-j  "  °j  be  less  than  or  equal  to 


2.0,  where  the  observations  are  number  of  personnel  required  to  operate  a 
given  unit  as  structured  by  a  computer  model.  Looking  at  Table  II,  It  Is 
seen  that  at  least  five  observations  (replications)  are  needed. Suppose  the 
following  observations  arc  obtained;  42.6,  41.8,  41.9,  42.1,  and  43.9. 
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Since  0^  -  41.8,  end  Og  «  43.9,  and  43.9  -  41.8  =  2.1  >  2.0,  more  observe- 

tlons  must  be  taken.  From  Table  II,  It  Is  seen  that  at  least  three  more 
observations  are  needed*  Suppose  that  the  next  three  observations  are  42.4, 
43.0,  and  40.7.  Now,  o2  ■  41.8,  o^  ■  43.0,  and  43.0  -  41.8  B  1.2  <  2.0. 

Therefore,  both  criteria  are  met,  and  sampling  ceases.  A  sample  size  of  eight 
observations  (replications)  has  therefore  been  found  to  be  adequate.  Suppose 
further  that  the  central  limit  theorem  Is  applicable  In  this  simulation 
analysis.  The  mean  It  42.3  and  the  median  Is  42.4.  Therefore,  42.3  Is  the 
mean  value  to  be  reported. 

Note  that  In  many  other  applications,  such  as  values  collected  In  surveys, 
the  median  Is  a  far  more  appropriate  measure  to  use.  It  would  limit  the 
Influence  of  any  outliers,  and  since  median  estimation  Is  the  focus  of  the 
theory  provided  here,  the  application  Is  exact. 
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FORMULATIONS  FOR  TABLE  I: 


COL.  1: 

aalf-axplanatory 

COL.  2: 

1  -  Cl)""1 

COL.  3: 

(0.8)n 

COL.  4: 

(0.9)n  s 

COL.  5: 

1  -  (n+1)  (h)n~l 

*  COL.  6: 

(0.8)n  *  0.2n  (0.8)n-1 

♦  O.Oln  (n-l)(0.8)"“2 

*  COL.  7: 

(0.9)n  +  O.ln  (0.9)"“l 

♦  0.0025n  (n-1)  (0.9) 

**  COL.  8: 

1  -  n<0.5)n“1  +  (n-1)  (0.5)n 

•*  COL.  9s 

1  -  n(0.75)n“l  ♦  (n-1)  (0.75)fl 

*  (l-2p)n  +  2  pn  (1-2P)""1  +  p2n(n-l)  (l-2p)n“2 

whara  p  ■  0.10  for  column  6  and  0.05  for  column  7 

•*.  l-n*n_1  +  (n-l)yn 

whara  V  ■  0.5  for  column  8  and  0.75  for  column  9 
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Appendix  B 


Finally,  not*  th*  following:  Suppos*  nj  obstrvttlons  ar*  taken,  and  thtn 
n2*n|  additional  observations  ar*  taken  In  th*  event  that  th*  first  n^  have  an 

unacceptable  rang*.  Let  PCo1SMSon]  *  P£©2SMl©  j].  This  Is  the  procedure 

recommended  In  this  paper.  It  Is  or  Interest  to  note  that  the  probability 
of  saying  the  median  has  been  bounded  It  larger  In  this  two  step  method  than 
if  all  n2  observations  were  taken  at  once.  This  Is  explained  In  th* 

unlikely  event  that  the  first  n^  observations  have  an  acceptable  range, 
but  th*  first  n2  (n2>n^)  ar*  such  that  °n  -j  “  ®2  *00  However,  If 

the  first  nj  are  acceptable,  It  Is  more  likely  that  these  n^  values  ar*  closer 

to  the  true  median  than  any  other  value.  This  Is  analogous  to  applying  a 
chi-square  goodness-of-flt  test  and  calculating  a  very  small  chi-square 
statistic.  There  may  be  a  very  small  probability  of  having  that  value  under 
the  null  hypothesis;  however,  there  Is  an  even  smaller  probability  of  having 
such  a  value  under  any  alternative  hypothesis.  Thus,  the  null  hypothesis  Is 
the  most  likely  candidate.  Similarly,  here  It  would  have  been  appropriate  to 
quit  after  n^  such  observations  as  described  above.  The  true  median  is 

probably  Indicated,  although  It  was  not  likely  that  this  would  happen  when  the 
underlying  population  probably  was  more  disperse  than  Indicated  by  such  an 
early  finish  to  this  sequential  procedure.  Thus,  one  could  stop  after  the 
first  step  with  a  savings  In  sampling,  because  the  median  may  be  located  more 
quickly  than  expected. 

The  above  considerations  affect  the  number  of  observations  needed;  however, 
It  does  not  affect  table  II  which  gives  the  probability  that  the  true  median  Is 
bounded  by  two  specific  values  found  1h  the  sample.  The  considerations  of  this 
appendix  deal  with  the  probability  that  th*  specific  values  will  be  close 
enough  at  some  step  along  the  way. 
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REALM  A,S,C,D,XN,BUN,ANS 

OPEN ( I , FILE* 'FAC, OAT* 'STATUS* 'OLD*, PISP* 'KEEP* ) 
OPEN( 1 2, F1LC*'FAC«0UI', STATUSR*NBN* ,DISP**KCIP*) 
tOO  READ<I,10,SND«1000)N,J 
10  rOARATCH,IJ,lX,II> 

XXN*N 

SUMO 

ANS*0 

00  100  1*1 ,  J 
ARN+i-I 
CALL  PACT(A) 

1*1-1 

CALL  PACT(I) 

C*1,0/(AM> 

sunrsuntc 

too  CONTINUE 
XN*N 

CALL  PACTl(XN) 

Oil, 

ANf*t •■XN*0*SUN 
NRITC(1»,14)N,J,ANS 
IS  F0RNAT(/,4X,2(1X,I)) ,BX,f4,4,/) 

00  TO  too 

tOOO  CONTINUE 

STOP 

END 

C 

SUBROUTINE  PACT(D) 

REALM  Of  PROD 
MOD-1, 

DO  100  I*1,D 

IX*D-IM 

PRQ0*PR0D*1I 

PROOtPROIVIO, 

tOO  CONTINUE 
OaPROO 
RETURN 
END 
C 

SUBROUTINE  PACTUM 
REAL*!  0,PR9D 
PROD-1, 

DO  tOO  1*1,0 

IMD-IM 

PR00*PR<1DMI 

PROORPROD/in, 

100  CONTINUE 
DRPROO 
RETURN 
END 
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DIUMIIDN  Y(B030) 

mi 

REXD(«,*)NC,J1 
If(NC,CE, 9000)00  TO  99 
J2aNC*Jl 

*■* 

DO  10  Jal,N 
NKaO 

DO  20  Mi 'PC 
YCKllMNd) 

lftYm.OT. 0.0)83  TO  20 

NXaNXtl 

CONTINUE 

ir((NK,QE.Jl).»ND.(NX,tE,J2))XaX4l. 

CONTINUE 

Pax/E 

MRITE(0,*)N,!1 
NRITE18,«)NC,JI,R 
NRXTE(7 ,*)N, 11 
NRITE17 ,*)NC, J1  #9 
XaO, 

90  TO  1 

•TOR 

END 
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QUANTILE- INFORMATION-FUNCTIONAL  STATISTICAL  INFERENCE 
AND  UNIFICATION  OF  DISCRETE  AND  CONTINUOUS  DATA  ANALYSIS 


Emanuel  Parzen 
Department  off  Statistics 
Texas  A&M  University 


ABSTRACT .  This  paper  presents  results  from  our  research 
program  "on  the  development  of  statistical  methods  based  on 
quantile-information-function  estimation  statistical 
inference.  Its  major  goal  is  to  define,  and  to  apply  the 
comparison  quantile  function  D(u;F,  G)  of  two  distribution 
functions  F  and  G.  We  outline  how  standard  methods  of  fitting 
a  parametric  model  F(y,0)  to  one  or  two  random  univariate 
samples  can  be  developed  in  terms  of  the  empirical  quantile 
function  Q“  (u)  by  suitably  defining  the  sample  comparison 
quant iLe  function  D~(u,0)  -  D(u;F~ ,F(» , 0)  and  the  sample 
comparison  density  function  d~  (u; 0)  -  D“'(u;0)  for  both 
continuous  and  discrete  data.  In  the  continuous  case  we  define 
D"(u);0)  -  F(Q~(u)}0).  In  the  discrete  case  we  define  first 
d"(u;0)  and  then  define  D"(u;0)  to  be  its  integral.  Other 
concepts  defined  include:  continuous  versions  of  discrete 
quantile  functions,  identification  quantile  function, 
information  distributions,  and  scientific  statistical  science. 
Emphasis  is  placed  on  developing  a  notation  which  applies  to 
both  discrete  and  continuous  data  analysis. 

KEY  WORDS-  Quantile  function  quantile  density  function, 
ilTFormatTon,  entropy,  function  estimation  functional 
inference,  comparison  quantile  function,  comparison  density 
function,  identification  quantile  function,  goodness  of  fit. 
probability  model  identification  scientific  statistical 
science. 

1-  PROBABILITY  MODEL  IDENTIFICATION  AND  COMPARISON  QUANTILE 
FUNCTIONS  . .  ~  “  '  “  “ 

Given  data  which  one  regards  as  a  random  sample  (of  size 
n)  of  a  random  variable  Y  one  would  like  to  infer  the 
probability  law  of  Y. 

Ensemble  Probability  Laws-  The  probability  law  of  Y  is 
described  in  general  by  its  "distribution  function 
F(y)-PROB[Y  <  y] ,  -•  <  y  <  •,  and/or  its  quantile  function 

Q(u)  *  F_1(u)  defined  by 

Q(u)  ■  inf  (y:  F(y)  _>  u} .  0  <  u  <  1 


Research  supported  by  the  U.  S.  Army  Research  Office  Grant 
DAAG29-83-K-0051  . 


Sample  Probability  Laws  (Nonparametrlc  Estimators) .  The 
first  step  in  statistical  data  analysis  bT'a“ random  sample  is 
to  form  tne  sample  distribution  function 

F" (y)  -  fraction  of  sample  <  y. 

and  the  sample  quantile  function  Q“(u)  “  F"  1  (u) .  The 
probability  theory  required  to  develop  statistical  theory  based 
on  F~(y)  and  Q"  (u)  is  given  in  Durbin  (1973)  and  Csbrgo  (1983) 
respectively. 

Parametric  Probability,  Laws.  One  approach  to  identifying 
F(y)  and  Q(u)  is  to  assume  "a  parametric  family  of  distributions 
F(y;0)  indexed  by  a  parameter  6  which  is  an  m-dimensional 
vector  belonging  to  a  parameter  space 

This  paper  develops  a  general  definition  of  the  comparison 
quantile  function  D(ujF.G)  of  two  distribution  functions  F(y) 
and  G(y).  The  ensemble  or  population  comparison  quantile 
function  is  defined  D(u;0)  ■  D(uj  F,  F(»  s  8))  . 

The  maximum  likelihood  estimator  §  can  be  shown  to  be 
definable  as  minimizing  a  measure  of  distance  between  F(yj0) 
and  F"  (y)  or  equivalently  the  distance  between  the  sample 
comparison  quantile  function  D“(u.0)  ■  D(u;F“.F(»  8)  and  the 
uniform  distribution  D0(u)  -  u. 

2 .  GOODNESS  OF  FIT  AND  COMPARISON  QUANTILE  FUNCTIONS 

"The  role  of  a~sirap IV "compar 1 s on  quantile  functions  is  best 
introduced  by  considering  the  goodness  of  fit  problems  test 
the  hypothesis  Hg:F(y)  -  F(y,0°)  for  some  specified 
parameter  value  8°. 

When  F(ys  0°)  is  continuous  one  transforms  Y  to  Z  - 
F(Y<e°),  called  the  probability  integral  transform.  Then 
Ho  is  equivalent  to  Hq;Z  is  uniform  on  the  interval  0  to  1 . 

When  the  sample  consists  of  observations  Y(1) . Y(n)  one 

transforms  to  Z(j)  ■  F(Y(j);0°).  Let  G“  (u) ,  0  <  u  <  1, 
denote  the  sample  distribution  function  of  Z (1 ) , . . . ,Z (n) .  The 
Kolraogorov-Smirnov  statistic  for  testing  Hq  is  defined  by 


D  -  /n  sup  | F“  (y)  -  F(y;0°) 

n  _»<y<« I 

or 


D  ■  /n  sup  |G~(u)  -  u| 
n  0<u<1  1  1 
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One  can  show  that 


D  ■  /n  sup  |G“*’1(u)  -  u| 
n  0<u< i  1  1 

We  propose  that  the  sample  quantile  function  G""1(u)  is 
the  most  useful  way  to  express  statistics  to  test  Ho*  One 

reason  for  this  is  the  elegant  formula  expressing  GM~1(u)  in 
terms  of  the  sample  quantile  function  Q"(u)  of  V(l),,  .  Y(n) 
and  the  comparison  quantile  function. 

G" " 1  (u)  -  D^ujeo)  . 

To  make  this  formula  concrete,  let  us  consider  the 
realistic  case  of  samples  described  by  empirical  probabilities, 
using  notation  such  as  the  following: 

K,  number  of  distinct  values  in  the  sample, 

V (1 )  <*..<  V (K) ,  ordered  distinct  values  in  the  sample; 

NV(J),  number  of  sample  members  equal  to  V(J),  for 

J-1 . . *K 

PV(J)  -  NV(J)/n.  empirical  probability  of  V <J) ; 

FV  (J)  -  PV(1)  +  ..  *+PV(J)  .  FV  (0)  -  0. 

Note  that  F"(V(J))  -  FV(J)  and  FV(K)  -  1. 

The  sample  quantile  function  QM(u)  has  the  elegant 
formula: 

<T(u)  -  V(J),  FV  (J-1 )  <  u  <  FV  (J)  for  J-1...K. 

The  sample  comparison  quantile  function  has  the  formula 

D~(u  6)  -  F(V(J),6),  FV(J-I)  <  u  <  FV(J)  for  J«1  . .  ,K. 

To  calculate  the  Kolmogorov-Smirnov  statistic  Dn  let 
Z  (J)“F(V(J)  ;  0°) .  Then 


Dn  -  Vn  max  max  {|Z(J)-FV(J)|  .  |z  (J)-FV(J-I )  | } 

To  perform  a  goodness  of  fit  test  one  needs  both 
numerical  and  graphical  procedures.  A  rejection  region  for 
Hq  at  the  95%  significance  level  for  large  sample  sizes  n  is: 


Dn  >  1.36  for  a  simple  hypothesis,  and  D«  >  .89  if  the  null 
hypothesis  is  the  composite  hypothesis  of  a  normal  distribution 
with  mean  and  variance  estimated  from  the  sample.  A  numerical 
procedure  compares  the  value  of  Dn  with  these  critical 
values.  A  possible  graphical  procedure  is:  plot,  on  the  same 
graph,  F"(y)  and  F(y;0°).  More  insight  is  obtained  by 
plotting  D  (u}6°)  and  D0(u)  *  u.  In  addition  many 
additional  numerical  procedures  for  testing  Hq  can  easily  be 
defined  in  terms  of  D  (u;6°)-u.  For  example  one  might  use 
the  Wilcoxon  type  statistic 

WL  -  /T?  {D“(u,e°)  -  u}  du 


Quick  graphical  procedures  for  identifying  the  probability 
law  of  a  random  sample  are  provided  by  QMI(u).  the  sample 
identification  quantile  function  defined  below  [section  4]. 

3_. _ CONTINUOUS  VERSION  OF  DISCRETE  QUANTILE  FUNCTIONS 

foments  can  ^e"  ex‘press e"d">el‘egahtTy  Tn  terms  of  Q  (u) ;  thus 

Mean  M  «  /^  Q(u)  du, 

Variance  VAR  -  Jq{Q(u)-M}2  du. 


Standard  deviation  DS  -  /VAR  . 

We  use  a  notation  for  mean  and  standard  deviation  which 
provides  analogies  for  measures  of  location  and  scale  based  on 
the  quantile  function 

Median  MQ  ■  Q(0.5)  , 

Quartile  deviation  DQ  -  2  f Q C • 7 5 )  -  Q(.25)}  . 

Our  concept  of  quartile  deviation  (which  equals  twice  the 
inter-quartile  range)  is  motivated  by  the  concept  that  in  the 
case  that  Q(u)  has  a  derivative  Q’(u).  the  derivative  Q '  (0.5) 
is  a  useful  universal  measure  of  scale;  DQ  is  a  difference 
quotient  which  can  be  regarded  as  a  very  rough  approximation  to 
Q'  (0.5). 


An  important  step  in  our  program  for  unifying  discrete  and 
continuous  data  analysis  is  to  associate  to  a  discrete  quantile 
function  a  continuous  quantile  functon  which  is  used  to  define 
the  median  and  quartile  deviation  of  a  discrete  distribution. 
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The  distribution  function  and  quantile  function  of  a 
discrete  random  variable  Y  are  piecewise  constant.  For 
example  let  Y  be  Bernoulli  with  PROB[Y  -  0]  -  q. 

PROBIY  -  1]  -  p,  where  p+q  -  1.  Then  F(y)  -  0,  q,  1  according 
as  y  <  0,  0  <  y  <  1.  1  <  y.  Further  Q(u)  -  0,  1  according  as 

0  <  u  <  q,  q  "<  u  <  1  .  We  propose  a  method  of  associating  with 
a  discrete  quantili  function  Q(u)  a  continuous  quantile 
function  denoted  QC(u),  which  for  the  Bernoulli  distribution 
has  value  0  at  u  ■  q/2,  value  1  at  u  ■  1-(p/2),  is  a  line  of 

slope  2  between  these  points,  and  has  value  P  u  -  0.5. 

Consequently  the  median  MQ  -  p.  in  agreement  with  the  fact  that 

the  mean  M  *  p. 

We  use  the  following  notation: 

K,  number  of  discontinuity  points  of  F(y)j 
V(1  )<. .  ,<V(K)  values  at  which  F(y)  Jumps, 

FV  (J)  -  F (V  ( J ) )  j  FV  (0)  -  0i  PV(J)  -  FV  (J)  -FV (J-1 )  \ 

U  (J)  -  FV ( J )  -  (PV  ( J )  /  2 ) . 

We  call  UF (J) ,  J-1...k,  the  midranks  of  the  discrete 
distribution  F(y);  they  play  a  (two-)  key  role  in  data 

analysis.  We  call  V(J).  J-1 . K.  the  probable  values  of  the 

discrete  distribution. 

The  discrete  quantile  function  Q(u)  is  given  by 

Q(u)  -  V(J).  FV  (J-1 )  <  u  <  FV  (J)  . 

The  continuous  quantile  function  QC(u)  associated  with  a 
discrete  quantile  function  Q(u)  is  defined  to  be  piecewise 
linear  between  its  values  at  u  -  UF(J)  where  it  is  defined  to 
satisfy 

QC(UF(J))  -  V(J),  J  -  1 .  ..K. 

For  0  <  u  <  UF  (1 ) ,  define  QC(u)  -  V  (1  > ;  for  UF(K)  <  u  <  1  , 
define  QC(u)  -  V(K). 

For  a  discrete  quantile  function,  we  define  its  median  and 
quartile  deviation  by 

MQ  -  QC (0.5) .  DQ  -  2  <QC(.75)  -  QC (. 25) ) 

The  mean  and  variance  of  the  midranks  U  are  denoted  MU  and 

K  K  , 

VARU:  MU  -  l  U(J)  PV(J),  VARU  -  \  {U(J)-MU}2  PV(J). 

J-1  J-1 


•  m  1  *  “  »,*  - 

.v. 


Always  MU  -  0.5.  VARU  is  approximately  1/12  (the  variance  of  a 
uniform  distribution  on  0  to  1). 


4. 


IDENTIFICATION  QUANT ILK .FUNCTION 

- to  a . quaTtTrrfuncnbrTTO)  .  one  associates  a  quantile 

function,  denoted  QI(u).  We  call  QI (u)  the  identification 
quantile  function  because  its  values  near  0  and  1  and  its 
overall  shape  can  be  shown  to  provide  quick  heuristic  methods 
for  identifying  the  type  of  distribution  (normal,  exponential, 
etc.)  and  its  tail  behavior  (short  tail  medium  tail,  long 
tail;  within  medium  tail,  one  can  distinguish  medium-short, 
medium-medium,  medium-long).  An  excellent  discussion  of  these 
concepts  is  given  by  Schuster  (1984). 


Wheh  Q(u)  is  continuous,  we  define  QI(u)  ■  {Q (u) -MQ}/DQ. 
When  Q (u)  is  discrete,  we  define  QI(u)  ■  (QC(u)  -MQ}/DQ.  One 
may  similarly  define  the  sample  identification  quantile 
function  cO‘(u). 

The  definition  and  applications  of  QI(u)  was  pioneered  by 
Parzen  (1984)  under  the  name  of  informative-quantile  function. 

Transforming  a  random  sample  by  subtracting  its  mean  M, 
and  dividing  by  the  standard  deviation  DS,  is  equivalent  to 
forming  the  Z-Quantile  function  QZ(u)  -  (Q(u)-M}/DS. 

5.  COMPARISON  QUANTILE  AND  COMPARISON  DENSITY  FUNCTIONS 

CONTINUOUS  CASE  ... 

- lTt~5Ty)  and  F(y)  be  continuous  distribution  functions. 

We  define  the  comparison  quantile  function  to  be 

D(u;F.G)  -  GF’ 1 (u) . 

Measures  of  distance  between  D(uiF,G)  and  D0(u)  -  u  provide 
measures  of  distance  between  F(y)  and  G(y). 

Although  the  mathematical  definitions  require  no 
interpretation  for  G  and  F,  we  usually  think  of  G(y)  as  a 
probability  model  for  Y  and  think  of  F(y)  as  the  true 
distribution  function  of  Y.  The  quantile  function  of  Z  -  G(Y) 

is  GF“l(u). 

The  quantile  density  d(u,F,G)  ■  D'(u;F,G)  is  called  a 
comparison  density  function.  Explicitly, 

d(u;F.G)  -  gF‘l(u)  /  fF" 1 (u)  . 

Its  interpretation  is  more  evident  by  writing  it  as  a 
likelihood  ratio: 

d(u;F,G)  -  g(x)/f(x)  when  F(x)  -  u. 


6.  INFORMATION  DIVERGENCE,  ENTROPY,  CROSS  ENTROPY 

WR*rTT~ind  G  are  Voth  continuous,  with  respective 
probability  density  functions  £(y)  and  g(y),  information 
divergence  is  defined  by 

I(F|0)  -  [-log  (g(y)/f  (y)>)  f(y)  dy 

Information  divergence  has  a  fundamental  decomposition 
I (F|G)  -  H(FjG)  -  H(F) 

defining  cross-entropy  H(F|G)  and  entropy  H(F)  by 
H(F;G)  -  /*„  {-log  g(y) }  f (y)  dy 


R(F)  -  H(F,F)  -  {-log  f  (y) }  f(y)  dy. 

It  should  be  noted  that 

R(F)  -  H(Q)  -  {-log  fQ (u) )  du  -  J]0  {log  q(u) )  du 

where  q(u)  -  Q' (u)  is  the  quant lie- density  function. 

An  Important  measure  of  the  distance  between  D(uiF,G)  and 
D0(u)  ■  u  is  the  negative  of  the  entropy  of  d(u;F,G)i 

-H(d)  ■  -  /q  log  d(u>F,G)  du. 

By  making  a  change  of  variable  u  -  F(x),  x  -  Q(u)  one  can 
verify  that  -H(d)  -  I(F;G).  This  fact  provides  a  new 
Interpretation  of  Information  divergence  as  a  measure  of 
distance  between  D(u;F,G)  and  u.  Minimising  Information 
divergence  I(P;G)  is  equivalent  to  maximizing  entropy  H(d). 

2j_  COMPARISON  DENSITY  DISCRETE  DISTRIBUTIONS 

Let  and  G(y)  be  "discrete  distributions  with  the  samo 
probable  values  V (1 )<. , ,<V (K) .  Their  probability  mass 
functions  are  denoted  PF(J)  and  PG(J)  respectively.  In  terms 
of  FV (J)  -  F (V ( J ) )  and  GV(J)  -  G(V(J))  we  define 

PF(J)  ■  FV (J)-PV (J-1 ) ,  P0(J)  -  GV  (J) -GV  (J-1 ) . 

We  define  the  comparison  density  function  d(u;F,G)  by 

d(uiF.G)  -  PU(J)/PF(J)  for  FV(J-I)  <  u  <  FV(J). 


We  define  the  comparison  quantile  function 


D(u;P,G)  -  Jjj  d(t}F .0)  dt 

As  an  example  of  how  d(u;F.G)  provides  a  unified  notation 
for  concepts  that  usually  are  defined  separately  for  continuous 
and  discrete  random  variables,  consider  the  Information 
divergence  of  discrete  distributions 


I  (F;G) 


One  may  Verify  that 


t-log  {PG(J)/PF(J)  }]PF(J) 


1 (F;G)  -  {-log  d (u, F,G) )  du 
The  chi-squared  statistic  is  interpreted  by 

!  uwkttflu!.  /;  u(u,f,g)-i]»  au. 

J"1 

Empirical  probabilities  are  denoted  PFM(J).  The 
asymptotic  distribution  theory  of  chi-squared  statistics  can  be 
expressed  as  properties  of  the  stochastic  process  D(utFN,G)-u, 
assuming  PG(J)  is  the  true  probability  mass  function. 


true  probability  mass  function. 


8.  WILCOXON  STATISTICS 

An important  measure  of  the  "distance"  of  D(ujF.G)  from  u 

is 

WL(F;G)  -  -Jq  {D(uiF.G)  -  u)  du 

The  Wllcoxon  statistic  used  in  two-sample  nonparametric 
statistical  Inference  has  mathematical  similarities  to  WL;  we 
therefore  call  WL  a  Wllcoxon  statistic. 

By  integration  by  parts  one  may  show  that 

WL(F;G)  ■  /q  u  d(u;F,G)  du  -  0-5 

When  F  and  G  are  both  discrete,  WL  has  an  Important  inter¬ 
pretation  in  terms  of  midranks- 
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^F(J-I)  U  du>  CJ>/PF(J)> 


i  K 

/'  u  d(uj  F.G)  du  l 
0  J-1 

K 

-  I  PG  (J)  UF  (J) 

J«1 

where  we  write  UF(J)  for  the  raidranks  of  F(y)  and  UG(J)  for  the 
midranks  of  G(y), 

We  can  interpret  WL  as  a  difference  of  means  of  midranks 
UFi 


WL  -  Eg[UF]  -  Ep  [UFJ 

One  can  also  show  that 
WL  -  Eg[UGJ  -  Ep[UG] 

9 .  BIVARIATE  AND  TWO  SAMPLE  PROBABILITY  LAW  MODELING 

techniques  '’ey  comparing  unlvar  ia^isT'dls  tfrTHut  Ion  '  functions 
can  also  be  used  to  model  the  relations  between  several 
variables  X  and  Y.  Relations  between  Y  and  X  can  be  expressed 
as  relations  between  the  conditional  distribution  functions 
FY iX (y : x)  and  the  unconditional  distribution  function  F(y).  We 
use  :  rather  than  to  denote  conditioning  and  we  define 

FY :X (y : x)  «  PROB  [Y  <  y  ;  X  -  x]  . 

The  concepts  and  algorithms  of  conditional  distributions 
can  be  Interpreted  to  apply  even  when  X  and  Y  are  not  random 
variables.  An  example  of  this  approach  is  the  problem  of 
comparing  two  samples. 

The  problem  of  the  comparison  of  the  distributions  of  two 
random  variables  Y^  and  Y2  given  random  samples 

Y 1  (1 ) . Y^np  and  Y2  (1 ) , . . .  Y2(n2)  can  be  formulated  as  a 

comparison  of  empirical  conditional  distributions  with 
empirical  unconditional  distributions.  Let  n  -  +  n2 . 

Define  n  observations  (X(1 )  ,Y(1 (X(n)  ,Y(n) )  of  a  pair  of 
hypothetical  random  variables  X  and  Y  by: 

(X(j).Y(j))  -  ( 1  .Y1  CJ) )  if  H  -  1 . n, 

-  (2 ,Y2(J-n1 ) )  if  j  -  n1  +1 . n  . 
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The  unconditional  distribution  of  Y  is  the  empirical 
distribution,  denoted  F~Y,  of  the  pooled  sample 
Y.j  (1 ) . Y1  (n1 ) ,  Y2(1),..,  Y2(n2).  The  conditional  distribu¬ 

tion  of  Y  given  X-1 ,  denoted  F“Y  X  "1 ,  is  the  empirical 
distribution  of  the  first  sample  Yj (1 ) , . . . ,Y^ (n^ )  , 

In  terms  of  the  comparison  distribution  function 
D~(u)  -  D(u  F~Y,F"Y!X  -  1) 

one  can  express  the  Kolroogorov-Smirnov  two-sample  statistic  as 
essentially  the  aupremem  of  |UN(u)  -  u  ,  and  the  Wilcoxon 
two-sample  statistic  as  essentially  the' Integral  of  D"(u)  -  u. 

One  can  give  explicit  formulas  relating  various  statistics 
which  one  forms  in  a  comprehensive  data  analysis  computer 
program,  and  which  are  in  fact  equivalent  to  the  Wilcoxon  rank 
sum  statistic.  Let 

S  -  sum  of  ranks  of  first  sample  in  pooled  sample, 

M  -  mean  of  pooled  sample  mldranko  UF“Y  in  first 
sample  (M  -  conditional  mean  of  UF*Y  under 
F**Y  :X  -  1){ 

RU  -  rank  correlation  of  midranks  UF^Y  and  UF“X. 

(for  truly  bivariate  samples,  RU  la  the 
Spearman  rank  correlation  statistic). 

One  can  show  that; 

M  -  (S/n^ n)  -  (1  /2n) 

(RU)2  -  (M-0.5)a(12n1/n2)  . 

10.  INFORMATION  DISTRIBUTIONS  AND  APPROXIMATIONS  TO  t  AND  F 

mmmzm - — - 

Another  aspect  of  the  unification  made  possible  by  a 
quantile-information-functional  approach  is  that  statisticians 
may  be  able  to  approximately  compute  siglflcsnce  levels  of 
standard  statistical  tests  without  having  to  consult  a  large 
array  of  statistical  tables. 

To  test  Ho  i  Y  is  N(pfo2),  p  «  0  against  Hi  :Y  is 
N (p. o2)  one  should  use  the  log  likelihood  ratio  test  statistic 
log  A  for  testing  p  ■  0  versus  p  *  0.  which  can  be  shown  to 
satisfy  [see  Rohatgl ,  (1984),  p.  723,  723] 
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-2  log  A  ■  n  J  ,  J  ■  In  (1  +  ^  • 

The  distribution  of  JM  is  called  an  information  distribution. 

It  can  be  described  symbollically  by  writing 

J**  *  In  (1  +  tJ.j) 

where  tRta^  denotes  a  random  variable  obeying  a  t  distribution 
with  (n-1)  degrees  of  freedom. 

The  sample  correlation  coefficient  R~  of  a  bivariate 
normal  sample  has  the  property  that  the  log  likelihood  ratio 
test  statistic  for  testing  R  -  0  against  R  a  0  satisfies  [see 
Rohatgi,  (1984),  p.  724] 

-2  log  A  -  -  n  In  (1-R"a)  -  nJ“ 

defining  J**  ■  -  In  (l-R"'2)  .  The  distribution  of  J"*  is 

an  information  distribution  which  can  be  described  symbolically 

J“  -  In  (1  +  —y  ta_2)  . 

Asymptotic  distribution  of  information  distributions.  One 
can  show  that  asymptotically 

n  In  (1  +  1  tj)  ♦  xj  .  chi-square  1  d.f. 

More  generally,  Let  n  denote  a  random  variable  obeying  an  F 

distribution  with  degrees  of  freedom  (d£)  equal  to  m  and  n  in 
the  numerator  and  denominator  respectively.  Then 

n  In  (1  +  -  Fm  n)  +  x*  .  chi-square  m  d.f. 

To  obtain  a  finite  sample  approximation  to  the 
distribution  of  the  information  distribution  we  write 

(*)  n  ln  o  +5  -  *><»•»> 

where  h(ra,n)  (which  tends  to  1  as  n+«)  is  a  constant  given 
explicitly  by 

h(».n)  -  h  -  |  (1  -  (1  +  n?j)'2/m))‘’ 
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For  n  large,  h  -  — .  For  m  -  1 ,  h  -  . 


The  constant  h  is  determined  by  equating  the  means  of  both 
aides  of  the  representation  for  F  Implied  by  eq.  (*) ; 


®  r*» 

n  in,n 


e  -1 


which  determines  h  to  satisfy 


The  high  degree  of  accuracy  of  formula  (*)  in  the  case 
m  «  1  is  discussed  in  a  paper  by  Gaver  and  Kafadar  (1984). 

If  one  uses  the  fact  that  the  t-dlstrlbutlon  enjoys  a 
close  approximation  (for  n  >  7)  by  the  distribution  of  x*/nh 

then  one  need  never  consult  tables  of  the  t-distrlbution  to 
form  tests  of  the  hypothesis  Hq:  \i  -  0.  One  can  directly 
Interpret  without  any  table  lookup  the  statistic  defined  by 


NJ'  -  (3.84) /h (1  .n-DJ** 


Reject  Hq  at  the  5%  level  if  NJ"  <  1  .  The  interval 
.5  <  NJ"<  1.5  corresponds  linearly  to  levels  of  significance 
.995  >  c  >  .905.  The  statistic  NJ"  is  useful  for  sample  size 
determination;  the  minimum  sample  size  at  which  the  observed 
value  of  J"  would  reject  the  hypothesis  that  J  -  0  is 
approximately  equal  to  NJ“  times  the  observed  sample  size. 


11.  SCIENTIFIC  STATISTICAL  SCIENCE 

This  paper  has  presented  results  from  our  research  program 
on  the  development  of  new  statistical  methods  based  on 
quantile-inforroation-functional  statistical  Inference  (which  we 
abbreviate  FunStAT  and  call  Functional  STatlstlcal  Analysis 
Technology).  The  overall  goal  of  this  research  program  is  to 
contribute  to  the  development  of  scientific  statistical 
science. 


We  propose  that  statistical  science  be  called  "scientific" 
(as  opposed  to  "artistic")  when  it  attempts  to  develop 
statistical  methods  in  a  unified  way  that  can  be  systematically 
applied  in  many  different  fields  of  statistics.  Scientific 
statistical  science  is  desirable  for  its  elegance  (in  order  to 
help  dispel  the  impression  among  applied  scientists  and 
engineers  that  statistical  reasoning  is  at  most  a  bag  of 
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tricks).  Scientific  statistical  science  is  desirable  for  its 
utility,  since  it  enables  statisticians  to  adapt  statistical 
methods  (in  order  to  develop  for  different  problems  innovative 
methods  customized  to  the  unique  features  of  each  problem). 

Scientific  statistical  science  may  be  a  necessity  in  the 
emerging  era  of  "PC/StAT",  personal  portable  computing 
statistical  analysis  technology.  Statistical  computing  for 
personal  computers  can  be  developed  to  be  interactive  for  ease 
of  use  and  for  effective  integration  of  classical  and  currently 
emerging  styles  of  statistical  data  analysis.  The  high  cost  of 
data  relative  to  computing  makes  it  sensible  and  wise  to 
analyze  one's  data  from  as  many  points  of  view  as  possible.  A 
unified  framework  for  statistical  reasoning  will  make  it 
possible'  to  more  rigorously  combine  the  results  yielded  by 
different  algorithms  which  are  applied  to  the  same  data. 

Examples  of  identification  quantile  functions  and 
comparison  quantile  functions  generated  by  our  IBM  PC  program 
EPSTAT  are  given  in  a  separate  report  available  from  the 
author. 
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THE  DISTRIBUTION  OP  A  WEIGHTED  VISIBILITY 
MEASURE  ON  A  LINE  SEGMENT  UNDER  SHADOWS 
CAST  BY  RANDOM  DISKS  HAVING  A  BIVARIATE 
NORMAL  SCATTERING* 
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ABSTRACT 

The  present  paper  develops  formulae  for  the  computations 
of  moments  of  a  weighted  measure  of  visibility  along 
short  line  segments  in  the  plane ,  when  the  shadowing 
objects  consist  of  N  (fixed  number)  of  disks,  which 
are  randomly  scattered  around  the  origin,  according  to 
a  bivariate  normal  distribution.  These  moments  are 
based  on  visibility  probabilities  of  points  on  the  line 
segment.  Formulae . for  the  computation  of  these  prob¬ 
abilities  are  derived.  These  formulae  are  based  on  the 
probabilities  of  general  rectangles,  half  circles  and 
triangles  when  the  points  have  a  bivariate  normal  dis¬ 
tribution.  The  appendices  provide  formulae  and  FORTRAN 
subroutine  functions  for  the  computation  of  the  required 
functions.  An  approximation  is  given  to  the  distribution 
of  the  random  measure  of  weighted  visibility.  The  results 
of  this  research  are  applicable  in  various  areas.  In 
particular  in  the  evaluation  of  the  performance  of  laser 
range  finders  and  other  similar  problems,  when  random 
objects  in  the  field  cast  their  shadows  on  a  target. 


Key  Wotidsi  Visibility,  weighted  measure  of  visibility , 
multinomial  random  ield ,  s  had  owing  •  ffco  ess , 

•  f'Ao  babilities  0(5  comvex  sets  unde  A  bivariate 
normal  distributions . 
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1.  Introduction 


The  present  paper  is  motivated  by  the  following  actual  problem. 

A  laser  beam  is  oriented  from  a  source  at  a  point  (xQ, yQ)  to  a 
line  segment,  T,  whose  center  is  located  at  the  point  • 

The  energy  in  the  beam  is  maximal  at  its  center  and  is  tapering 
off  fast  as  the  distance  from  the  center  increases.  A  common 
model  for  the  distribution  of  the  intensity  of  the  energy  around 
the  center, of  the  beam  is  the  spherically  symmetric  Gaussian 
distribution,  with  very  small  standard  deviation.  Certain  portions 
of  the  beam  may  be  obstructed  by  obstacles  which  are  randomly  dis¬ 
persed  in  the  field.  These  obstacles  could  oonsist  of  different 
types  of  objects  which  are  in  the  field,  such  as  trees,  bushes, 
piles  of  dirt,  etc.  The  ray  does  not  penetrate  through  such 
objects.  If  such  an  obstructing  object  intersects  any  ray  from 
<x0,y0)  to  T,  we  say  that  the  object  casts  a  shadow  on  T.  There 
may  be  different,  sometimes  overlapping  shadows  which  are  cast  on 
T  by  different  objects  in  the  field.  Thus,  certain  points  on  T 
are  in  the  "light"  and  certain  ones  might  be  in  the  "dark".  Let 
(xR,yR)  denote  the  right  hand  limit  point  of  T  and  (x^y^  its  left 
hand  limit  point.  Let  A  ■  xR-xL.  We  assume  that  the  intensity 
of  a  ray  connecting  (xQ,y0)  with  a  point  (x  ,  y  )  on  T  is  proportional 
to  the  normal  (Gaussian)  probability  density  function  (PDF).  The 
intensity  function,  w(x),  is  normalised,  so  that  its  integral  from 
x^-A  to  x^+A  is  equal  to  1.  We  define  an  integrated  measure  ,  W, 
of  the  random  amount  of  light  (energy)  that  reaches  T  from  the 
source.  W  is  a  random  measure  having  a  distribution  which  depends 
on  the  characteristics  of  the  random  field  of  the  shadowing  objects. 
In  our  previous  studies  (Yadin  and  Zacks  [3,4])  we  discussed 
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properties  of  random  visibility  measures  when  the  shadowing  objects 
are  disks  of  random  radii  and  random  location.  Furthermore,  we 
assumed  that  the  number  of  centers  of  disks  falling  within  any 
specified  (Borel)  sat,  B  ,  in  the  plane  has  a  Poisson  distribution 
with  mean  X (B) .  The  radii  of  diskB  may  or  may  not  depend  on  the 
location  of  their  centers.  Such  a  model  is  called  a  Poisson 
random  field  of  shadowing  objects.  The  present  study  follows  a 
different  scattering  model.  First,  we  assume  that  the  number  of 
disks,  N,  is  fixed  (finite).  Furthermore,  given  any  partition  of 
the  plane  to  (Borel)  sets  B^,  ...,  Bm,  the  number  of  disks  J^,  ...,  , 

whose  centers  belong  to  B^,  ...,  Bm,  respectively,  have  a  multi¬ 
nomial  distribution,  with  probability  veobor  (ir^,  . ..,  um)  which 
depends  on  the  specified  sets  and  on  the  stoohastio  scattering 
mechanism.  In  addition  each  disk  has  a  radius  which  is  a  reali¬ 
zation  of  a  random  variable  with  a  specified  distribution.  Such  a 
model  is  aalled  a  multinomial  field  of  shadowing  objects. 

In  the  present  paper  we  further  assume  that  the  centers 
of  disk  have  coordinates  which  are  independent  random  vectors 
having  a  given  bivariate  normal  distribution,  and  that  the  radii 
of  disks  are  independent  and  identically  distributed  random  variables, 
independent  of  the  center  locations.  The  motivation  for  studying 
such  a  model  is  due  to  a  particular  military  application,  in  which 
the  shadowing  objects  are  artillery  rounds.  N  rounds  are  scattered 
according  to  a  bivariate  normal  distribution  around  an  aim  point 
(x*,y*).  Each  round  when  exploded  creates  a  cloud  of  dust  and 
debris  of  random  size,  A  planar  cut  of  such  a  cloud  is  modeled 


as  a  disk.  The  model  of  the  present  paper  can  be  further  generalized 
to  cases  of  several  clusters  of  N  disks,  each  one  characterized  by 
a  different  bivariate  normal  distribution. 


In  Section  2  we  formally  present  the  structure  of  the  field 
and  the  probability  models  of  the  associated  random  variables.  In 
Section  3  we  discuss  methods  for  computing  the  probability  that 
any  specified  point  on  T  is  in  the  light,  and  the  probabilities 
that  any  n  specified  points  of  T  are  simultaneously  in  the  light. 
These  are  called  visibility  probabilities.  The  visibility  prob¬ 
abilities  are  required  for  the  computation  of  the  moments  of  the 
random  measure  W.  In  Section  4  we  discuss  some  properties  of  the 
distribution  of  W  and  the  computation  of  its  moments.  A  beta- 
mixture  approximation  to  the  distribution  of  W  is  discussed  in 
Section  5.  In  Section  6  we  discuss  the  subroutines  which  were 
programmed  for  computing  the  moments  of  W  and  the  parameters  of 
the  beta-mixture  approximation.  Numerical  examples  are  presented 
in  Sections  3-5.  As  shown  in  Section  3,  the  computation  of  the 
visibility  probability  of  a  point  requires  subroutines  to  compute 
the  probabilities  of  arbitrary  rectangles  and  half-circles,  under 
a  standard  bivariate  normal  distribution.  The  determination  of 
the  simultaneous  visibility  probability  of  n  points,  n  >  2,  requires 
subroutines  for  the  computation  of  probabilities  of  arbitrary 
triangles  under  standard  bivariate  normal  distributions.  Formulae 
of  these  probabilities  are  derived  in  Appendices  A  and  B.  A. R. 
Oidonato  and  R.  K.  Hageman  [2]  published  a  method  for  computing 
the  probability  of  an  arbitrary  polygon  under  a  bivariate  normal 
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distribution.  Their  method  is  based  primarily  on  the  computation 


of  probabilities  of  angular  sectors.  The  leading  factor  in  such 

2 

computations  is  exp(-R  /2)  where  R  is  the  distance  of  the  vertex 
of  the  angle  from  the  origin  (the  center  of  the  bivariate  normal 
distribution) .  The  main  formula  cannot  be  applied  if  R  is  large 
and  some  asymptotic  approximations  are  provided.  As  shown  in  the 
examples,  we  are  interested  in  triangles  having  at  least  one 
vertex  which  is  very  far  away  from  the  origin  (R  >  100).  The 
direct  calculations  which  are  presented  in  the  appendioea  do  not 
require  computations  of  angular  probabilities,  as  in  Didonato  at  al 
[2] .  The  subroutines  provided  in  the  present  paper  yield  very 
aacurate  results  in  a  matter  of  a  few  seconds  of  computer  time. 

2.  The  Theoretical  Model 

Let  Pc-(x0,y0)  denote  the  source  of  the  light  (laser)  beam. 

Let  T  be  a  line  segment  of  width  w  ■  2 A  centered  at  -  (xt,yt) 

and  perpendicular  to  the  line  segment  I’  Furthermore,  let 
PR  -  (xR,  yj  and  PL  ■  (xL,yL)  denote  the  right  and  left  end  points 
of  T  respectively.  Thus,  if  bt  denotes  the  slope  of  the  line 
through  P0  and  Pt,  the  coordinates  of  PR  and  PL  are  given  by 


(2.1) 


-  xt  +  A|bt|/(l+bJ) 


2*1/3 


xL  -  xt  -  A|bt|/(l+b2)1/2 


yR  ■  yt  +  sgn(bt)A/(l+b2)1^2 


yL  ■  yt  -  sgn(bt)A/(l+b2)1^2 
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where  sgn(bt)  is  the  sign  of  the  slope  bt<  Let  D^,  D2,  . 


N 


be  N  random  disks.  Bach  random  disk  is  characterized  by  the 


triplet  (Xi,  Yj.,  Ri),  i  -  1, 


•  •  •  # 


N 


,  where  (X^,  Y^)  are  the 


coordinates  of  the  center  of,  the  disk  and  is  its  radius. 
We  assume  that  (Xjy  Y^) ,  . . . ,  (xN,  yn 


Yn)  are  independent  and  identi¬ 
cally  distributed  (1ID)  random  vectors  having  a  bivariate  normal 
distribution  centered  at  the  origin  (0,0).  Without  loss  of 
generality  we  assume  that  the  bivariate  normal  distribution  has 
zero  correlation,  i.e.,  p-0 .  Indeed,  if  pi*0,  one  can  apply  the 
orthogonal  transformation  (X' ,  Y* )  ■  (X,Y)(B),  where  (B)  is  a  2x2 
matrix  consisting  of  the  eigenvectors  of  the  covariance  matrix 
of  (X,  Y) .  The  orthogonal  transformation  B  maps  the  random  disks 
into  random  disks  having  the  same  radii.  Furthermore,  the  dis¬ 
tribution  of  (X<  ,  Y')  is  bivariate  normal  centered  at  (0,0) 
with  correlation  sero  and  variances  X1  and  Xj#  respectively  where 
X.^  and  X2  are  the  eigenvalues  of  the  cover ianoe  matrix 


$ 


PVy 


pox°y 


These  eigenvalues  are 

.2 


°x  +  °v 

5 


•+  i 
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k  - 


+  4p2o 


(2.2) 
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(0x  °y)  +  4p  Vy 
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where 


The  elements  of  B  are  ^ ,  i,  j  ■  1»2  , 

blj  “  SJ  c5x0y 

(2.31  b2J  ’  5J .<*?  *  \)>  > 

Oj  -  toj  -  X.V1  +  P2»*oJl1/a  •  1  -  !«*• 


Th«  point*  (X0.y0)«  (*K«yR)  »na  <*LiyL)  *r*  tt»n»fom*d  to 


corresponding  points  U0»n0)»  {5R#nR)»  UL#nL)«  The  line  segment 
T  is  traneformed  to  the  line  segment  T 1 »  connecting  UL»nL)  with 

The  radii  of  the  random  disks »  Rj^  ...»  Rj,#  are  asaumed  to  be 
independent  random  variables,  having  a  common  distribution!  0R(r), 
oonoentrated  over  the  interval  [a,b],  where  0  <  I  <  b  <  "•  It  ip, 
also  aaaumed  that  {R^»  i*"l»  ...»  N)  are  independent  o£  { (x^,y^)  ji*!# 
The  randomly  scattered  diaks  may  oaat  shadows  on  T.  In  Figure  1 
we  illustrate  a  possible  realisation  of  such  a  random  shadowing 
process ,  in  whiah  N  -  4  diaks  are  randomly  scattered.  Two  disks 
cast  shadows  on  the  line  segment  T.  Thus,  creating  two  dark 
intervals  at  the  edges  of  T  and  one  visible  interval  in  the  middle. 
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Lit  F  -  (x,y)  bi  «  point  on  T,  and  lit  I(x)  ■  1  or  0  if  P  is 
in  thi  .light  (viaibli)  or  in  thi  dark.  Thi  intigrmtid  measure  of 
visibility  on  T  is 


(2.4) 


x  +A 

'  x-x*. 


1  f 

w  "  at  J  ♦  (^ri)  I(x) 


dx 


V* 


L  A 


■  j  /  ♦  (u)I(xt-wu)du  , 

-A/t 


whiro  0 < t <®  is  thi  standard  dsviation  of  thi  dispersion  of  the 


beam,  and  <|>(u) 


-maL-  ixp(-u2/2),  -*<u<»,  is  thi  probability 
vzir 
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density  function  (PDF)  of  the  standard  normal  distribution. 
Moreover,  A  *  2*(£)-l,  where  $(*)  is  the  standard  normal  integral. 

Notice  that  W  is  a  random  variable  assuming  values  in  the  interval 
[0,1].  W  -  0  if  T  is  completely  in  the  dark  and  W  *  1  if  T  is 
completely  visible.  The  distribution  of  W  has  two  jump  points, 
at  0  and  at  1.  In  the  open  interval  (0,1)  the  distribution  of  W 
is  absolutely  continuous.  Let  F^(w)  denote  the  cumulative  distri¬ 
bution  function  (CDF)  of  W  and  let  pQ  -  Pr{W*-0},  pj.  -  Pr{W-l),  then 


(2.5) 


Fw(w) 


po+(1"po"pl*  /*w(y*dy  '  for  °-w<1 


for  w>l. 


where  gw(w)  is  the  PDF  of  the  absolutely  continuous  component  of 
(w)  . 

.  Visibility  Probabilities 


In  the  present  section  we  develop  formulae  for  the  probabilities 
that  rays  connecting  PQ  with  n  (n>l)  specified  points  on  T  are  not 
intersected  by  any  one  of  the  N  random  disks.  The  methodology 
always  follows  the  following  algorithm! 

S.l.  Determine  the  set  B(r)  of  (x,y)  points  such  that, 
if  a  random  disk  of  radius  R«r  has  a  center  (x,y) 
in  B(r)  then  at  least  one  of  the  n  specified  rays 
is  intersected  by  the  disk. 

S.2  Determine  the  probability  of  B(r),  i.e. 

Pr(<0(r)}-  /  /  «xp(-  -j(x2+y2)  Jdxdy  . 

B(r) 


„•  v  v  v  v  ..vwv  .v.v.-'  V-V*  1  ■  •  v  W/Vj’ 


S.3.  Compute  the  total  probability 


b 

H  »  j  Pr(B(r)}dGR(r) . 

a 

S.4.  Compute  the  simultaneous  visibility  probability  of  the 
n  points. 

\\>  -  (1-H)N 

3.1  The  Visibility  Probability  of  a  Single  Point  on_T. 

Let  P  ■  (x,y)  be  a  point  on  T.  Let  PQP  be  the  line  segment 

connecting  PQ  with  P.  The  set  B(r)  io  the  union  of  a  rectangle 

C  tr)  and  two  half  circles  C,(r)  and  C,(r)  (see  Figure  2).  Two 
o  * 

sides  of  CQ(r)  are  parallel  to  apd  at  distance  r  from  it.  The 
half  circles  Cx(r)  and  C2<r)  are  each  of  radius  r,  and  are  attaohed 
to  CQ(r)  at  the  sides  perpendicular  to  FJJjp.  In  Appendix  A  we 
provide  the  formula  for  the  computation  of  Pr(B(r)}in  the  oase 

i 

that  GB(r)  is  a  uniform  distribution  on  [0,b].  Let  (x0,y0»x,y) 

denote  the  H- function  corresponding  to  the  present  case.  This 

function  provides  the  probability  that  a  random  disk  intersects 

FT.  The  visibility  probability  of  the  point  P  is 
o 

(3.1)  (x0,yc#x,y)  -  (l-H1(x0,y0,x,y))N  . 

Notice  that  for  large  values  of  N, 

(3.2)  t|>3  (x0,y0,x,y)*exp(-A1(x0,y0,x,y))  , 

where  A1(xQ,yo,x,y)«NH1(x0,yorX,y) 
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Consider  three  point*  P0"(X0»Y0) '  and  p2"*x2'y2* 

in  the  plane.  Let  T<x0'Y0»xi*Yi»x2,y2*  denote  the  bivariate 
normal  probability  that  a  random  vector  (X,Y)  belong*  to  the  *et 
inscribed  by  the  triangle  AP^^Pj.  We  adopt  the  convention  that 
PQ  is  the  vertex  having  the  smallest  y  coordinate,  and  Pj^Pj 
are  reached  from  PQ  in  a  counterclockwise  direction.  In  Appendix 
B  we  provide  the  formula  for  determining  T(x0,y0,x^,y^,x2,y2) t 

The  set  B(r)  corresponding  to  the  simultaneous  visibility  of  two 
points,  Pj^  and  P2  on  T  is  shown  at  Figure  3. 
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FIGURE  3.  The  Set  B(r)  for  the  Simultaneous  Visibility  of 
aHBT P». 

Let  P^(r)  *  (x£(r),  y^(r))  ,  i-0,1,2  ,  be  the  vertioee  of  the 

triangle  AP*(r)P^(r)P2(r) .  The  aides  of  this  triangle  are 

parallel  to  those  of  APoPlP2  ttn^  at  d^*tarvo*  r  *rom  them. 

AP+(r)pt (r)pt  (r)  is  an  outer-triangle  of  distance  r.  Similarly,  we 
o  x  * 

define  an  inner-triangle  of  distance  r,  AP0(r)P^[(r)P2(r) .  The 
inner  triangle  is  contained  in  AP0P1P2»  with  sides  parallel, 
respectively,  to  those  of  APQP^P2  an^  *t  distance  r. 


■  ,v.v.v»vy>v 


*/.  v,v*v 


5TV 

ftv 


The  set  B(r)  corresponding  to  the  points  P^Pj.  and  P2  is  contained 
in  the  set  APG(r)p£(r)P2(r)-APG(r)P^(r)P2  (r) ,  which  is  the  r-outer 
triangle  minus  the  r-th  inner  triangle.  If  the  distance  of  the 
point  PQ  from  the  origin  is  greater  than  4  the  probability  of  B(r) 
is  given  to  a  high  degree  of  approximation  by 

(3.3)  Pr{B(r) }  *  T (x* (r) ,y£ (r) ,x£ (r) ,y£(r) ,x2 (r) ,y2 (r) ) 

~  T(x"(r)  ,y~(r),x^(r)  ,y][(r)  ,x“(r)  ,y2(r)) . 

We  remark  here  that  in  certain  instances  the  inner  triangle  does 
not  exist.  In  such  oases  we  say  that  the  inner  triangle  is  an 
empty  set,  and  its  probability  ii  zero.  We  further  remark  that 
in  the  many  actual  applications  the  source  of  light  is  more  than 
4  units  away  from  the  origin,  and  formula  (3.3)  provides  excellent 
approximation.  The  conditions  under  which  the  inner  triangle  does 
not  exist  is  given  in  formula  (3.4)  below.  The  H-funqtion 
corresponding  to  (3.3)  will  be  denoted  by  **2  (x^y^x^y^x^yj) . 

This  function  is  evaluated  by  numerical  integration.  For  the 
determination  of  x^(r) ,y|(r) (i*0,l,2)  and  of  x^(r),y^(r)  (i*0,l,2), 

we  distinguish  between  five  cases t 

Case  Ii  x2<xo<xl  * 

Case  Hi  x,<x."*x„  , 

2  1  O  J 

Case  lilt  x,j<x,<x„ 

2  10 

Case  IV i  xG  ■  x2<x1? 

x^<x.<x, 

0  2  1 


Case  Vt 


Let  y=a^+b^x  be  the  formula  of  the  line  passing  through  PQ  and  , , 

let  y=a2+b2x  be  the  formula  of  the  line  passing  through  P^  and  t,. 

and  let  y»a3+b3x  be  the  formula  of  the  line  through  P2  and  PQ. 

4* 

The  line  passing  through  PQ(r)  and  P1(r)  has  in  Case  I  the  formula 

Y'^-r (1+b2)  1/,2+b^x.  Similarly,  the  line  through  P*(r)  and  P2(r) 

2  1/2  + 

has  in  Case  I  the  formula  y»a3-r(l+b3)  +b^x  .  xQ(r)  the 
x-coordinate  of  the  point  of  intersection  of  these  two  lines. 

In  this  manner  one  can  obtain  explicit  formulae  for  the  coordinates 
x^(r),  y£(r),  i«0,l,2;  and  cf  x][(ri,y^(r).  Thepe  formulae  are 

listed  in  Appendix  C.  In  all  the  above  five  cases,  the  condition 
under  which  the  r- inner  triangle  is  empty  is 

(3.4)  a2+r(l+b^)1/2+b2x“(r)<y^(r). 

3 , 3  Probabilities  of  Simultaneous  Visibility  of  n  Points  on  T. 

Let  »  i"l»  n,  be  n  points  on  T,  such  that 

xn<xn_1<  ...  <  x^.  Generalizing  the  results  of  the  previous 
section,  the  probability  of  B(r)  is  given  approximately  by 

(3.5)  PrtB(r)}*  TOc+'U*)  ,y*(r)  ,x*(r)  ,y*(r)  ,x*(r)  ,y*(r)) 

n-1 

-  ^  T(*o,i<r>'yo,i(r>.xI(r).yi(r).xi+1(r),y1+1(r)) 
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Notice  that  some  or  all  the  r-inner  triangles  might  be  empty  and 
the  corresponding  T-f unctions  are  then  equal  to  zero.  The  condition 
under  which  the  i-th  inner  triangle  is  empty  is 


(3.6)  an+r(l+bj)  1^2+bnx^i(r)  <  y“fi(r)>  '  i“1'  •••'  n“1, 

an  and  bn  are  the  intercept  and  slope  of  the  line  passing  through  P,  and  Pn« 

The  coordinates  x”  .^{r)  and  y“  ^(r)  can  be  obtained  in  the  following 
manner.  Consider  the  triangle  AP0Pj.P,j_+^.  The  coefficients  a2  and  b2 

in  the  formulae  of  Appendix  C  should  be  replaced  by  and  bn. 

a3  and  b^  should  be  replaced  by  the  intercept  and  slope  of  the 

line  through  PQ  and  Pj^,  i.e.,  and  bi+^.  Finally,  the 

H- function  corresponding  to  (3.5)  is  denoted  by  Hn(*c0,yo,x1,y1, 

. xn,yr).  The  probability  that  n  points  are  simultaneously 

visible  is 


(3.7) 


Vxo'yo'xl'yl'-*>'Vyn)  "  <1-Hn(xo'yo'’**'xn'yn)) 


N 


EXAMPLE  3.1 

In  the  present  example  we  illustrate  some  of  the  visibility 

probabilities.  The  parameters  of  the  calculations  aret 

N-20  x  *0  x  -0 

o  t 

o  -1  y  -  -33.3  y  -  -.333 

x  ■*  o  t 

o-l  d-. 167 

GR(r)  is  unif  orm  on  [0,b]. 


b-. 333 


We  consider  k“7  triplets  on  T,  having  coordinates  (x^ 

(Xi2.lt*  and  *xi,3'yt*'  wher® 

x,^  3  -  -  A+iA/4  ,,  x^^2  ■  iA/8  and  ■  A  ,  i  ■  1,...,7. 

Let  *1(1  -  ♦i(*0.Vxi.3'»t> 

*2,1  “  *2<Wxi,].'*t'xi,3'*t> 

♦j.i  •  i,3<xo'yo'xi,i^t'xi,2'yt'xi,3'yt)> 

The  values  of  these  visibility  probabilities  axe> 


i 

*2,1 . 

' *3,1  ... 

1 

.3080 

.1233 

.1108 

2 

.3064 

.1360 

.1262 

3 

.3053 

.1516 

.1439 

4 

.3052 

.1692 

.1642 

5 

.3055 

.1927 

.1885 

6 

.3064 

.2201 

.2186 

7 

.3080 

.2539 

.2534 

Notice  that  iK  t  is  symmetric  around  the  origin  (1*4).  The 
simultaneous  visibilities  of  two  and  three  points  increase  as 
the  points  become  closer.  Notice  also  that  ^2,1  an<^  ^3fi  ®r® 
quite  close,  for  each  i,  and  that  their  difference  diminishes  as 
the  distance  between  the  three  points  decreases. 
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The  probability  that  the  line  segment  T  is  completely  visible 
is  determined  in  the  following  manner.  Let 

(3.8)  H5(xo'yo,slR'yR'xL'yL*  ■ 

b 

/  T(x*(r)  ,y*(r)  ,x£(r)  ,yR(r)  ,x£ <rl  ,y£(r) )dGR(r) . 

& 

h2 (x©'y0'xR'yR'xL'yL^  **  the  probability  that  a  random  disk  will 

cast  a  shadow  on  T.  Hence,  the  probability  that  T  is  completely 
visible  is 


(3.9)  Pj-<lHSJ<x0,yofxRfyR,xL,ylj))N 


4.  The  Moments  of  the  Random  Measure  W. 

The  k-th  moment  of  W  is  given  according  to  (2.4)  by  the  formula 

A/t 

(4.1)  yk  “1  E  {(  /  ♦  (u)l(xt+tu)du)k) 

A  -A/t 

k  k 

*  H  f"«f  n  +  (Uj  )G{  IT  I(xfe+TU,)}  dUi  , 

A*  i  J  i-1  1  i-l  z  1  x 

Sk 

where  (u^, . . .  ,uR)  t  -A/t iujt<ujt-i< •  •  •<u2<u1<A/t}  . 

Moreover 


(4.2) 


k 

e{  n  Kx.+tuJ) 
i-i  *  1 


*k(xo'yo'xt+Tul'yl'  •  •  • '  VTUkfyk) 


is  the  probability  of  simultaneous  visibility  of  the  k  points. 


Making  the  transformation  z^  *  ^  u^,  i«l,  ...»  k,  the  k-th  moment 
of  W  can  be  expressed  as 

1  11 

<M|  *k>  J  <M^*k-l>  /  •**  /  ♦<$*!>• 

zk  2k-l  *2 

h  (W V4zi'yi'  •  *  *  'V4  dzr  •  *dzk 

Notice  that  y^a+BAz^  (i*l,  ...»  k),  where  a  and  B  are  the  intercept 

and  slope  of  the  line  segment  T.  In  particular,  the  expected 
value  of  W  is 

1 

(4.4)  vi i  ■  Jr  /  ♦  (^*)i|<1(xe,y0,xt+As,yt+BAs)d*. 

-1 


-1 


The  second  moment  of  W  is 


(4.5) 


V2 


7T~J  /  ♦'tV  /  ♦(f*1H’2(*o'yo'*t+4*l'yt+AS*l’ 
A  T  * 

-1  *2 


Xt+4z?/yt+4^z2*dzldz2  . 


These  integrals  have  to  be  computed  numerically,  as  discussed  in 

Section  6 .  The  computation  of  a  high  order  moment  according  to 

(4.3),  with  a  high  degree  of  accuracy,  is  time  consuming.  One 

can  obtain,  however,  lower  and  upper  bounds  for  Vk(k>3),  in  a 

manner  which  will  be  readily  discussed.  We  precede  this  discussion 

with  the  following  comment i  Aa  k+<»  vk  approaches  p^.  Indeed, 
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according  to  (2.5), 


1 

(4.6)  nk  -  (i""P0“P|j_)  J  yk9w(y)<*y  +  pr 

o 

Hence#  by  applying  the  Dominated  Convergence  Theorem,  Uk*Px 
k-*-». 

To  obtain  bound*  for  yk(k>3)  coneider  the  inequality 

(4.8)  ^3(xo'yo'xl'yl'xk'yk*  - 

\(x0,y0,xi,y1,x2,y2, . . .  ixk,yk)  < 

1<,2(xo'yo'xl'yl'xk»yk)  ' 

for  every  k>3,  where  \|»J  (x0,y0,x1#y1,xk#yk)  -  (1-H*(x0,y0,x1,y1,xk,yk) ) 

This  follows  direatly  from  (3.5)  And  (3.8) .  We  remind  that 
^2(xo'yo'xl'yl'xk'yk*  ia  th®  Probability  that  th*  interval 

between  (x^y^  and  (xk»yk)  is  visible.  Substituting  these  lower  and 

upper  bounds  of  ’l'k(x0,y0,x1,y1, . . .  ,xk,yk)  in  (4.3)  we  obtain  lower 

and  upper  bounds  for  yk<  Thus,  if  yk^  *nd  WkfU  denote  the  lower 
and  upper  bound  for  yk  one  obtains,  for  every  k>3, 


^2  *xo'yo'xt+Azl'yt+A0z.l'xt+Azk'yt+^l3zk*  * 


(4.9) 


is  obtained  from 


where  $(*)  is  the  standard  normal  integral,  vi. 

JC  f  u 

(4.9)  by  replacing  the  function  by  The  proof  that  sub¬ 

stitution  of  in  (4.5)  yielde  (4.9)  can  proceed  by  induction. 
Indeed,  for  k«3  we  obtain  from  (4.3) 


(4.10) 


3  1  1 

"3,i, -rrs  /  ♦•tv  /  ♦<K>  /♦<K>  »(*1.*j)a»i<s.,<a«, 

A  T  -1  a*  i. 


where  P(xlf*3)  -  ^Vyo' V*A*l'yt+Ae*l'xt+A*3'yt+A**3* 


Furthermore 


1  1 
A  f  A,  /A  m  \  f  A  I  A. 


(4.11)  7  J  4>(f  a2)  /  ♦(t*l)F(il'I3)dald*J 


■  f  /  <M“  xl)F(*1#a3)  /%<£  e2)da2dx1 

*3  *2 

1 

-  J  X1)F(X1,«3)  C*(^  ■3)Jd«1 

*3 

Substitution  of  (4.11)  in  (4.10)  yields  (4.9)  for  k-3.  For  k>4 
one  obtains  from  the  induction  hypothesis, 


(4.12) 


j  1  1  11 

15  /  <MTzj+l>  /  *( f*j)  /•*•  /  <M?*i)F(*1r*k)dx1...dx 


Zj-1  *2 


•m 

37  /  *i)p(»i.«kM*(|.1)-*(f,j+snj 


for  all  jBl,2|...,X|  £^k— 3,  that 


,'.v  v ,  v <-.v!  'Xv  i. 


.* 


(4.13) 


/♦'K+21/  ♦<£,w  /  •••  /  *<^i>F<vvav--d2 


*4+3  Z4+2 


■a+l  *2 


1  ,  1 


*4+3  Z4+2 


But, 


(4.14)  f  /  ♦(f«us)[»(f.1)-*(A.,+2)l*4«J 


■lil  !4<K>-*<7*t«>]m 


Substituting  (4,14)  In  the  right  hand  aida  of  (4.13)  wa  obtain 

1  11' 

.4+1  r  ,  ,  . 

(4.15)  J< |>(f*A+2)  J  •••/  ♦(f**1)F(*1,Bjc)dB1d*2. .  .dEAH 

*4+3  *4+2  Z2 

1 

•m  /  *(^1>F<zi'V‘4l7Ii>-*<7Ii+3»lMldIi 

*4+3 

In  particular,  for  4»k-3,  (4.3)  and  (4.15)  yield  (4.9). 


Example  4 . 1 

In  the  present  example  we  illustrate  numerically  the  bounds 
for  the  first  10  moments  in  a  given  case.  The  first  two  moments 
are  computed  exactly  according  to  formulae  (4.4)  and  (4.5).  The 


bounds  to  the  moments  of  order  k>3  were  computed  according  to  (4.9).  The 
parameters  of  the  computation  are: 

°x  ‘  °y  ■  1 
xQ  -  333 

v  «  -100. 
o 

Xt  “  0 

yt  *  -.333 

A  -  .167 
T  -  a/3 

and  a  ■  0,  b  »  .333. 

The  first  two  moments  are  *  .3037  and  v>2  ■  .2375.  The  lower  and 
upper  bounds  for  k-3,  ...,  10  are  given  in  the  following  table. 


A  good  estimate  of  can  be  obtained  as  the  midpoint  between 
the  lower  and  upper  bounds.  The  limit  of  as  k**»  is  p^.0855. 
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5 .  Approximating  the  Distribution  of  W  and  Determining  pQ . 

The  CDF  of  W  is  a  mixture  of  a  discrete  and  absolutely  con¬ 
tinuous  component,  as  presented  in  formula  (2.5).  The  only  element 
of  that  formula  which  we  have  already  discussed  is  p1#  as 
specified  in  formula  (3.9).  We  provide  here  an  approximation  for 
P0  and  for  the  absolutely  continuous  component  of  Fw(w),  which  is 
called  the  beta-mixture  distribution. 

The  beta  CDF  is  represented  by  the  incomplete  beta  function  ratio 

(5.1)  VW  -  Blu^vjr  J  u  (1"u)  dU' 

where  0<v-L>v2«»,  and  B(v1,v2)  is  the  complete  beta  integral.  This 

two-parameter  family  of  distributions  can  represent  a  variety  of 
different  shapes  and  forms  of  distributions  on  the  interval  (0,1). 

We,  therefore,  approximate  Fw(w)  by  the  beta-mixture. 

(5.2)  Fw(w)  -  p0  +  (1-Pq-?!) lw(v1,v2)  ,  0<W<1 

and,  obviously  Fw(w)  -  1  for  x>l.  The  parameters  pQ,  and  \>2  of 

(5.2)  are  determined  by  equating  the  first  three  moments  of  (5.2) 

to  those  of  W.  The  solution  of  these  moment  equations  (see  [3])  is 

v1  -  (2y£-y^(y.j+y£)  )/Dat. 

(5.3)  v2  -  (y[-yj)  (y£-yJ)/D*t. 
and 

p0  -  i-pr*1i(VV/v1  ' 

2 

where  Det.  ■  y|y£-(y£)  . 


Example  5.1 

In  Table  3  we  provide  a  numerical  Illustration  of  the  beta- 
mixture  approximation,  under  the  parameters  0  «o  -1#  x  ■  -.333, 

a  y  u 

yo»  -100.,  xt*0, .333, .666,  1.000,  1.333,  -.333,  A-.167,  t-A/5 

and  a-0,  b».333, 


xt 

,J2 

w3 

Po 

Pi 

V1 

.000 

.3057 

.2375 

.2126 

.6928 

.0855 

53.3 

.333 

.3257 

.2570 

.2314 

.6728 

.0980 

52.9 

.667 

.3864 

.3170 

.2898 

.6120 

.1404 

52.2 

1.000 

.4854 

.4190 

.3914 

.5131 

.2261 

55.1 

1.333 

.5538 

.5287 

.3884 

.3633 

67.6 

TABLE 

3.  The 

First  Three  Moments  and  the 

Parameters  of  the 

Beta 

-Mixture  Distributions 

The  beta  distributions  with  v^>l  and  0<V2<:1-  have  PDF's  whiah  are 
monotonioally  increasing  from  0  to  »  .  They  are  called  J-shaped 
distributions.  In  the  cases  of  Table  3,  sinoe  is  very  largo, 
most  of  the  mass  of  the  absolutely  continuous  component  is  ooncen 
trated  near  the  value  W-l.  This  means  that  the  distribution  of  W 
is  almost  a  2-point  distribution.  For  example,  xt*“0,  W-0  with 
probability  pQ-.693  and  with  probability  .307  W>.95. 

The  values  of  (5.2)  in  the  case  of  xt"0  are  given  in 


Table  4. 


6.  Numerical  Compulation* 

The  numerical  computations  of  Tables  1-4  were  performed  on  an 
IBM-370  system.  The  programs  were  written  in  FORTRAN.  They  consist 
of  the  main  parts  and  the  following  subroutine  functions. 

1.  FUNCTION  VPT  (X$ ,  ,XT,  YT,B) 

2.  FUNCTION  TRL  (X$ , ,X1 , Y1 ,X2 , ¥2) 

3.  FUNCTION  SVP  (K,X<|>,Y4>,X1,Y1,X2,Y2,X3,Y3,B) 

4.  FUNCTION  TVP(X.«|>,Y$,X1,Y1,X2,Y2,B) 

,3.  FUNCTION  CNDX(Y) 

6.  FUNCTION  BETA(X,V1,V2) 

and 

7.  FUNCTION  GAMA (V) . 

Subroutine  functions  5-7  compute  the  standard  normal  CDF,  the  beta 

CDF  and  the  gamma  function,  respectively.  Subroutine  VPT(X<J>,Y$,XT,YT,B) 

computes  the  visibility  probability  of  the  point  (XT,YT)  from  the 

Ronrcc  (X<J>,Y<j>) ,  where  the  distribution  of  the  disks  radii,  R,  is 

uniform  (rectangular)  on  the  interval  (0,B).  The  subroutine  TRL 

(X$ , Y<)>  ,X1 , Y1  ,X2 , Y2 )  computes  the  standard  bivariate  normal  probability 

that  a  point  (X,Y)  falls  within  the  triangle  with  vertices 


P  - (X<J> ,  Y(j>) ,  P^=  (XI ,  Y1 ) ,  P2=(X2Y2)  *  p0  always  t^ie  P°^nt  with  the 

smallest  y-coordinate.  The  vertices  and  P2  are  reached  from 

P„  in  a  counter  clockwise  direction.  The  subroutine 
o 

SVP(K,X<J>,Y<|>,X1,Y1,X2,Y2,X3,Y3,B)  computes  the  probabilities  H2  or 
K3  for  the  simultaneous  visibility  of  k*2  or  k-3  points  (see 
formula  (3.7)).  K»2  or  K»3  is  the  argument  for  the  number  of  points, 

if  K«2 ,  the  only  arguments  which  are  taken  into  account  are 
X<j>,Y<f>,Xl,Yl,X2, Y2  and  B.  The  values  of  X3  and  Y3  can  be  arbitrary. 
The  points  PQ* (X<|>,Y<J>) ,  P1»(X1>Y1)  and  P2«(X2,Y2)  are  the  vertices 

of  the  triangle,  while  P3«*(X3,Y3)  is  a  point  in  the  line  segment 
connecting  P^  and  P2*  The  funation  TVP(X<f>,Y<j>,Xl,Yl,X2,Y2,B) 
computes  the  probability  (x0,y0,x1,y1,x2,y2) .  The  subroutine 
functions  VPT  and  TRL  were  programmed  for  the  standard  bivariate 
normal  model  with  zero  correlation,  i.e.,  a  bivariate  normal  dis¬ 
tribution  centered  at  the  origin,  with  o  -cr  *1  and  p-0.  In  actual 

*  y 

problems  the  variances  of  X  and  Y  are  generally  unequal.  In  such 

cases  we  make  the  additional  transformation  x+x/a  and  y+y/o  .  This 

^  y 

reduces  the  model  to  a  standard  bivariate  normal.  The  random  disks 
are  transformed  to  random  ellipses.  However,  we  replace  each  such 
random  ellipse  by  a  random  disk  having  a  radius  equal  to  the  large 
radius  of  the  corresponding  ellipse. 

The  moments  y2  and  the  lower  and  upper  bounds  for  u  ^  ( k  =^3 ) 
are  computed  according  to  formula  (4.9)  by  employing  the  subroutine 
functions  TVP  and  SVP  and  using  a  Gauss  quadrature  12-point  numerical 
integration  (see  Abramowitz  and  Stegun  [1]  pp.916).  The  FORTRAN 
programs  of  these  subroutine  functions  are  given  in  Appendix  D. 
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The  main  program  reads  the  parameters  of  the  problem  and  reduces 

it  to  the  standard  bivariate  normal  model  under  which  all  the 

computations  are  performed.  For  example,  suppose  that  a  source  of 

a  laser  beam  is  located  at  the  point  (-1,  -3000)  and  a  target  of 

width  1  is  centered  at  (0,-10).  N-20  disks  are  scattered  according 

to  a  bivariate  normal  distribution  centered  at  (0,0)  with  parameters 

o-3,  o-30  and  p-0.  The  radii  of  disks  are  uniformly  distributed 
x  y 

in  the  interval  (0,1).  The  transformation  x'-x/ox,  y‘-y b'-b/ox, 
w'— w/°k  reduceB  the  abovG  parameters  to  those  of  Table  3. 
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APPENDIX  A.  The  Bivariate  Normal  Probability  of  a  Rectangle  with 


Two  Half  Circles  and  The  Probability  tnat  a  Random 


Disk  Intersects  a  Line  Segment 


We  develop  here  formulae  for  the  computation  of  the  standard 
bivariate  normal  probability  of  the  set  B{r)  described  in  Figure  2. 
The  set  B(r)  is  the  union  of  a  rectangle  CQ(r )  and  two  half  circles 
C1(r)  and  C2(r),  having  a  radius  r.  The  rectangle  CQ(r)  has  two 
sides  which  are  parallel  to  the  line  segment  pQpt ,  where  P0n(x0,y0) 

and  Pt*(xt»yt) «  Moreover,  these  sides  are  of  distance  r  from 

'p  Pt .  We  provide  now  a  formula  for  Pr{C0(r)>.  We  distinguish 
between  three  oases i 

Case  I i  xt<xo  ' 

Case  II s  x.*x^  , 

to 

Case  III*  x. >x^  . 

to 

In  Case  II  we  have 


(A.l) 


pr*co  *  [•(xt+r)-*(xt-r)]  l*(yt)-*(y0)] 


where  4(z)  is  the  standard  normal  integral.  In  Cases  I  and  III  we 
consider  the  orthogonal  matrix 


( A.  2 ) 


1  B  -1 

1  B 


where  B-(yt~y0) / (xt-xQ)  is  the  slope  of  the  line  segment  PQPt 
It  is  easy  to  verify  that  the  orthogonal  transformation 


fy,y  4  xNi 

\x  V  \  y/ 


4  4  *  «  *  «  "  »  1  t  *  *  *  1 *,*  * 


*jr *  *  » *i  k. 


yields  points  P'  and  P'  having  the  same  x-coordinate.  Furthermore 
o  t 

y£>y '  if  xt>x0  and  y^y^  xt<xo'  Accordingly, 


t4»(x^+r)-<Kx^-r)  ]  IMy^)-*  (y£)  1  *  in  Case  I 


(A*  3) 


Px{CQ(r)} 


[♦(x^+r)-*(x^-r) ]  t* (y^) 1  .  ,  inCase  III, 


Indeed,  the  distribution  of  (X‘,Y')  ia  a  standard  bivariate  normal 
distribution,  with  correlation  p-0.  The  half  circle  C^r)  ia  centered 
at  (x£,y£)  and  C2(r)  is  centered  at  (x^,yj) .  Consider  Case  III, 
in  which  yj.>y^.  The  probability  of  C^tr)  is 

(A. 4)  Pr{C1(r)}»Pr{|X'-x4i<r,y^  <  *'  <  y£  +  (r2- (X* -x^) 2 ) 1/2} 

x£+r 

-  J  ♦  (x)  My4+(r2-(x-x4)2)1/2)dx  - 


xt~r 


4>  Cy^)  I*  (x^+rj-^ix^-r)  ] 


Similarly, 


(A. 5)  Pr.(C2<r)}  -  (My^J  [«(x4+r)-«(x4-r)l  - 


xt+r 


/  ♦'(x)*(y^-(r2-(x-x4)2)1^2dx 


xt-r 


From  (A. 3)- (A. 5)  one  obtains  in  Case  III, 


7m 


(A. 6)  Pr{B(r) } 


l  Pr{C, (r) } 
i-0  1 


*t+r 


-  /  <M*> [*<yi+(r2-(x-K^)2)1/2) 


xt"r 


-  ♦(y<J-(r2-«(x-x^)2)1/2)]dx. 


Formula  {A. 6)  applies  also  for  Case  IX,  by  sutetituting  for  i:£ 
and  yQ,yt  for  y^y{-  In  Case  I  we  interchange  y£  and  y£  in  (A. 6) 

2  2  1/2) 

We  remark  also  that  if  y^  <-4,  the  term  ♦<y0-(*  )  can 

,  •  •  ,  ‘ 1  "  T  ' 

be  dropped  from  formula  (A.C). 

Finally,  the  probability  that  a  random  disk  inter seot*  the 
line  segment  PQPt  is,  in  Cases  II  and  111, 


(A. 7)  Hl<x0#yo#xt#yt) 


b  *t+r 


|  /  /  «i,(x)[4(y4+(r2-tx-x4)2)1/Z)-®(yi-(r2-(x-x4)2)1/2 


o  xt-r 


1  1 

r»  r, 


-b  j u  ft  (x^+buz) (« (y;+bu (l-*2) 1/2 ) -* (yj-bu (1-z2) 1/2) ] 


o  -1 


The  double  integral  in  (A.  7)  was  evaluated  by  a  double  P -point 
Gaussian  integration  (see  Abramowitz  and  Stogun  [It  pp.916]. 
Attempts  at  20-point  integration  yielded  negligible  difference. 


) ] dxdr 


dzdu 


AFTENDXX  B.  The  Bivariate  Normal  Probability  of  Triangles, 
7<xo;yo/x1,y1<x2,y2) . 


Consider  the  triangle  AP0P^P2  '  wkara  pi"^xi'yi)  *  i“0,l,2. 

We  make  first  an  orthogonal  transformation  of  the  plane  (x,y)  so 

that  the  side  pJpJ  of  the  triangle  AP^P£P£  is  Paralial  to  the 

x~axis  and  y'«min(yj )•  The  orthogonal  matrix  applied  for  this 
0  o<i<21 


purpose  is: 


(B.l) 


*  ‘TT*5?75 


1  B 


L  -B  1 


,  if  Xj^Xj  , 


where  B— / (Xj-x^) .  It  Xj«x2  we  use  the  transformation 


0  -1 


1  0 


r  if  xQ  <  xx 


(B.2) 


0  -1 


-1  0 


,  if  xQ  > 


Let  (X' ,Y' )  -  (XfY)A' .  The  distribution  of  (x',y')  is  like  that 
of  (X,Y). 


> .  «v« v .  * 

.V 'i'.  , 


Let  be  the  standard  bivariate  normal  probability 

(p*»0)  of  the  triangle  AP^P^P£  .  Notice  that  y£*y£,  and  y^Y^ 

Purthermore,T(x0,y0,x1,y1,x2,y2)  -  T(x^,y^,x^,y^xJ,yJ) . 

We  distinguish  between  five  cases. 

Case  li  x£<x^<x£i 

Let  y'*a2+b2x'  be  the  line  connecting  P^  with  P£  and  let  y,«a1+b1x' 
be  the  line  connecting  P^  with  P£.  The  probability  of  AP^P.jP£  is  then 


(B.3) 


T<xo#yo'xl,yl#x2,y2)  " 

Pr{xJ<X'<x£  ,  a2+b2X'<Y'<yp  + 

Pr{xUX'<x.[  ,  a^^X'lY'iyp 

■  *(yj)  [♦(x[)-«(xj)]  - 

x'  X1 

J  ?(x)fl»(a2+b2x)dx  -  J  <Mx) 4>(a1+b1x)  dx  . 


The  integral  in  (B.3)  was  evaluated  numerically  according  to  the 


formula 
(B.  4) 


<Mx)$(a+bx)dx  «  Z  *(a+b5i)  I*(54)-*(C1.i)]  # 

i««l  1  1  11 


where  N«2Q,  are  the  endpoint  of  equal  subinterval  partition  of 


As  in  Case  1,  let  y»a1+b1x  be  the  line  connecting  PQwith  P^, 


Then, 
(B.  5) 


®<*o»yo,xl#yl'x2'y2)  " 

Pr{x^  <  <  x£  ,  a1+b1X'<i  Y’  <_  yp  ■ 

x ' 

[0 (xp-^tx^)]  -  J1<p(x)9  (a1+b1x)dx. 


Catejh  x£  >  x^. 


Let  y»a2+b2x  denote  the  line  connecting  PQ  with  P2  and 
y«a1+b1x  denote  the  line  connecting  PQ  with  P^.  Then 


(B.6) 


T<xo'yo'xl'yl'x2,y2)  “  *<y{) (♦(x[)-*(x£)] 

x2  xi 

+  4> (x) <t>  (a2+b2x)dx  -  J  $ (x)*U]+bjX)dx. 


Case  4:  x>xi 
. .  l  o 


As  before,  let  y-a2+b2x  designate  the  line  connecting  PQ  with 
P^,  then 


(B.  7 ) 


T<xo»yo,xl,yl,x2,y2)  "  *(yp  C*  <x^) (ac^)  ] 
-  J  ^(x)*(a2+b2x)dx. 


AND  OTHER  WRITTEN  HE 


y”(r)  ■  aj+r  (l+b^  ' 


*E^  lU+b^'Wb2)1'2) 


Yjtr)  "  *2+t  d+^2^^+*:52x2  ^  1 


Xj(r)  *  2x2“x2^  » 

yZ(r)  -  a2+r(H-b2)1/2+b2x2(r) 


Case  2t  x^x-l  . 

x*(r)  *»  xQ+r  „. 

y+(r)  -  a3-r(l+b2)1/2+b3x+(r)  , 


x0(r)  -  xQ-r  , 

y~  (r )  ■  a3+r  (l+b2)  1/^2+b3x0(r)  , 

x£(r)  -  x+r 

y^(r).  “  a2+r  (l+b2)  ^2+b2x^  (r)  , 


x^tr)  ■  x-r 

y“(r)  ■  a2+r  (l+bj) 'L^2'fb2xl ^  ' 
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a„-a 


x2(r>  a  ytzk:  +  +rhr  [d+b?)1/2+(i+b?)1/2] 


b3“b2  b3~b2 


YgCr)  -  a2+rU+b2)  1/2+b2xJ(r)  , 


x”(r)  ■  2x2-x2(r)  , 


Y2  <r >  "  *2,te(l+b2)1/2+b2x2<r) 


Case  3 1  x_  >  x, 
_  o  i 


xo(r) 


a, -a 


E^EJ  +  SpB ~  td+b2»1/2+(l+b2)1/2] 


y*(r)  -  a1+r(H*bJ)1/2+fc1xJ(r) 


xo(r)  ■  2V*o(r)  ' 


y“(r)  ■  a^rd+b2)  1^2+b1x“(r)  , 


xI<r)  "  E^  +  b^  ^ (1+b2)  1/2-(l+b2) 1^2] 


y^(r)  -  a2+r(l-i-b2)  1/^2+b2xJ(r)  , 


xn  (r)  -  2x^-x^  (r)  , 


y“(r)  -  a2+r(l+b2)i/2+b2x[(r) 


_  a3“a2  r  ...2,1/2.  .,.2,1/2, 

x2(r)  "  U~=r3  ■  E^  t(l+b2)  +(l+b3)  ] 
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y^(r)  “  a2+r (l+b^)  ^^+‘^>2X2 ^  ' 


x2(r)  -  2x2-x2(r)  , 


y2(r)  -  a2+r (l+b2) 1^2+b2x2(r) 


Case  4  >  xq-x2  . 


x+(r) 

m  xA-r 

0 

**0M 

-  •1-r{l+bJ)1^a+b1x*(r) 

i 

*;<rl 

■  x^+r 

0 

*‘0M 

-  a1-r(l+b2)1/2+b1x^(r) 

» 

x+(r) 

ftl"a2  r  ...St  1/2.  ..2*  1/2, 

■^=BJ  -E7E7tu+bl)  H1+b2>  1 

yj(r) 

-  a2+r(l+b2)1/2+b2x^(r) 

9 

1 

X 

«  Ixj^-x^r)  , 

y;<r) 

»  a2+r(l+b2)1/2+b2x][(r) 

9 

x2  (r) 

-  x2-r  , 

N! 
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yj(r)  o  aj+tU+bjr'SbjxJlr) 


t 


x2(r)  *  x2+r  , 


y2(r)  «  a2+r 1^2+b2X2(r) 


Case  5*  x_<x„ 

. . O  2 


a, -a 


*>'  ■  EpS*  *  CU+bJ)l/2+<l+b^)l/2] 


y*(r)  -  a^-r (1+b2) 1/2+b1x^(r)  , 


x“(r)  -  2xQ  -  x*(r)  , 


y~(r)  -  a^r  (1+b2)  1^2+b1x“  (r)  , 


l(l+b2)l/2+(X+bj)1/2l 


y^(r)  -  aj+rd+b^^+bjxJtr)  , 


x-!  (r)  ■  2x1-x^[(r)  , 


y^(r)  '■  a2+r(l+b2)1/f2+b2x^(r)  , 


.  a3_a2  r  , ,,  ,  i_2v  1/2  ,,.,.2,1/2 

x2(r)  ET^bT  +  E7ET  l  (l+bj)  -(l+b2) 

2  3  2  3 


x2(r)  -  2x2-xJ{r)  , 


y“(r)  »  a2+r(l+b2)1/2+b,x:(r)  . 


2>  '~2"2 
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ON  SEGMENTATION  OF  SIGNALS,  TIME  SERIES,  AND  IMAGES 


Stanley  L.  Sclove 

Departaent  of  Quantitative  Methods 
University  of  Illinois  at  Chicago 

ABSTRACT 

Signals  and  tlae  series  often  are  nob  hoaogentous  but  rather  are 
generated  by  aeohanlsas  or  processes  with  various  phases.  Slallarly, 
laages  are  not  hoaogeneon  but  contain  various  objsots.  "Segaenbation" 
Is  a  proosss  of  atteaptlng  to  reoovsr  autoaatloally  the  phases  or 
objsots.  A  aodel  for  representing  such  signals,  tlae  aeries,  and  laages 
Is  dlsoussed.  Soae  approaches  to  estlaatlon  and  segaentatlon  In  this 
aodel  are  presented. 


Key  words  and  phrases i  statistical  pattern  recognition, 
classification;  teaporal  correlation,  spatial  correlation;  optlalaatlon 
by  relaxation  aethod. 
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1 .  Introduction 

About  ten  years  ago,  Professor  Jamas  Osterburg,  a  oolleague  at  the 
University  of  Illinois  at  Chioago  end  an  expert  on  physloal  evldenoe 
(Osterburg  and  O'Hara,  194$)  Osterburg,  1968,  1982),  oonsulted  me  in 
regard  to  establishing  probability  estlsates  for  partial  fingerprints. 
He  have  had  a  very  interesting  collaboration  mhloh  resulted  in  several 
papers  (Osterburg,  Parthasarabhy,  Raghavan,  and  Solove,,  1979)  Solove, 
1979,  1980,  1981a)  and  may,  thanks  to  Professor  Osterburg 's  continuing 
efforts,  result  in  ohanges  in  praotioes  regarding  the  evaluation  of 
fingerprints  as  evldenoe.  For  the  present  paper  the  point  about 
fingerprints  is  thisi  When  at  professional  meetings  t  mould  talk  about 
the  subjeot,  oertaln  people  (usually  eleotrloal  engineers  or  oomputer 
scientists)  mould  tell  me  that  mhat  X  mas  doing  mas  "image  processing." 

The  reason  that  our  fingerprint  mork  resesb1.es  image  processing  is 
this.  Osterburg  treated  fingerprint  analysis  by  plaolng  a  grid  of  cells 
over  the  print.  One  then  notes  the  looations  of  any  ooourrenoes  of  the 
"Galton  details,"  the  minutiae  of  the  ridge  lines,  suoh  as  ridge  endings 
and  forks.  In  nuaerioal  image  processing  a  real  isage  is  divided  into 
cells  ("plxels"i  pioture  elesents)  and  one  notes  nuaer ioally  mhat 
ooours  in  eaoh  oell.  The  real  image  is  expressed  as  a  matrix  —  roms  by 
columns  —  of  oells,  just  as  the  TV  soreen  has  a  matrix  of  dots  mhioh 
are  illuminated  mith  various  oolors  and  intensities. 
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2.  Iaages 

An  important  aspect  of  inage  prooasalng  la  Image  "segmentation," 
the  location  of  objects  in  the  inage. 

Examples,  (i)  A  pioture  of  a  house  and  yard  is  to  be  labeled  using 
the  labels,  brick,  glass,  tree,  grass,  sky.  (ii)  An  image  is  to  be 
labeled  using  the  labels,  tank,  mud,  tree,  sky.  ( ill >  A  medical  image 
is  to  be  labeled  using  the  labels,  tumor,  or  normal  tissue. 

Segmentation  involves  labeling  each  pixel  aooording  to  the  name  of 
the  objeot  of  Mhioh  it  is  a  part,  in  turn  involves  the  grouping  of 
neighboring  pixels.  Slnoe  I  had  earlier  worked  on  oluster  analysis,  the 
grouping  of  observations  (Solove  1977),  it  was  natural  to  oonoentrate  on 
segmentation.  The  labeling  process  can  be  made  explioit  in  a  model 
which  states  that  at  eaoh  pixel  we  observe  the  value  of  a  random 
variable  X,  but  also  along  with  X  there  ia  an  unobservable  variable,  the 
label.  (I  tend  to  treat  the  labels  as  parameters.  Others  treat  them  as 
missing  data,  i.e.,  random  variables.)  In  the  oontext  of  this  model, 
segmentation  is  merely  estimation  of  the  labeling  parameters. 

The  random  variable  X  is  often  a  vector  of  several  measurements. 

Examples.  (1)  A  familiar  example  of  a  vector  of  measurements 
(though  it  would  not  be  measured  aoross  a  two-dimensional  array)  is 
blood  pressure,  which  is  a  veotor  of  the  two  measurements,  systolio  and 
dlastollo.  (11)  In  color  television,  X  is  a  veotor  of  three 
measurements,  the  red  level,  green  level,  and  blue  level,  (ill)  In 
Landsat  satellite  data,  there  are  four  spectral  channels,  one  in  the 
green/yellow  visible  range,  a  second  in  the  red  visible  range,  and  the 
other  two  in  the  near  infrared  range,  (lv)  For  a  blaok-and -white  image, 
the  random  variable  X  which  is  simply  a  scalar  consisting  of  a  single 
gray  'evel  measurement,  rar.her  than  a  veotor. 

Iaages  are  two-dimensional;  I  deolded  first  to  consider  a 
one-dimensional  version  of  the  problem.  An  image  is  a  two-way  series;  a 
one-way  series  la  a  "time  series."  So  I  began  this  research  by  thinking 
about  segmentation  of  time  series. 
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3*  Tine  Series 

The  problen  of  segnentafcion  considered  here  Is:  Given  a  tine 
aeries 

(x(t),  t*1,2,  ...  ,n}, 

partition  the  set  of  values  of  t  into  subseries  (segments,  reglnes)  for 
which  the  values  x(t)  are  relatively  honogeneous  in  sobs  sense.  The 
segnents  are  assumed  to  fall  into  several  classes. 

Examples.  (i)  Segment  a  reoeived  signal  Into  background,  target, 
background  again,  another  target,  eto.  (ii)  Segment  an  EEG  of  a 
sleeping  persion  into  periods  of  deep  sleep  and  restless  or  fitful  sleep 
(two  classes  of  segment).  (ill)  Segment  an  BCG  into  rhythmic  and 
arhythmio  periods  (two  olasses  of  segment),  (iv)  Segment  ah  economic 
time  aeries  into  periods  of  recession,  reoovery,  and  expansion  (three 
olasses  of  segment). 


Next,  several  simulated  time  series  mill  be  shown.  The  first  is 
relatively  smooth;  the  aeaond,  relatively  rough  or  jumpy.  (They  are 
simulated  first-order  autoregressions  with  autoregression  ooeffiolents 
equal  to  +.8  and  -.8,  respectively.) 


32.500  ♦ 


7  01 


4  7 

2  6  89 


Saooth  time  series  (simulated  first-order 
autoregression  with  coefficient  +.8) 
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Y(T) 

6 

32.500  +  6 

680  2840  9 

-2  24704  0  8 

■  5 

8  7 

30.000  +  34  5  2  23  6 

-  1  7  1  1 

5  3  7  4  0 

5  1  8 

369  57  9 

27.500  +  9  1  3 

9 

+ - +— - + - + - - — + - +  T 

10  20  30  40  50 

Rough  bint  series  (simulated  first-order 
autoregression  with  ooeffioient  -.8) 


The  next  series  has  alternating  saooth  and  Juapy  segaents. 


Y(T) 
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Slaulated  tlae  series  with  three  segaents 
(saooth,  then  rough,  then  saooth) 


It  is  a  slaulated  series,  with  three  segaents.  Each  segment  is  a 
first-order  autoregression.  For  t  *  1  to  20  the  ooeffioient  is  +.8;  for 
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t  s  21  to  30  the  coefficient  i&  -.8;  then  for  t  &  31  to  50  the 
ooeffioient  is  sgsln  +.8.  In  aotual  data  analysis  on*  would  not  know 
ths  sagaents  but  would  have  to  find  thea  and  estimate  their  parameters. 


Now  let  us  oonaider  a  real  example,  rather  than  simulated  data.  The 
graph  below  Is  quarterly  ONP  (Gross  National  Produot)  for  55  quarters, 
starting  with  the  first  quarter  of  1946. 

LOG  GNP 

2.700  +  45 

123 

34567890 

89012 

01  67 

2.550  t-  789  2345 

123456 

0 

9 

123  78 

2.400  +  890  456 

67 

-  345 

-  12 

10  20  30  40  50  60 

Plot  of  Y(T),  where  Y(T)  is  log  (base  ten)  quarterly  GNP  in 
billions  of  ourrent  (unadjusted)  dollars.  T»1  is  1946-1,  T*2 
is  1946-2,  ...,  T»5  is  1947-1,  eto.  (N«55) 

Here  is  a  second  difference  of  the  log  quarterly  GNP.  Do  you  see 

anything  particularly  Interesting? 
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1000D1D4 

w» 

3*500  + 


4 

3  5 


12 

3  0  3 

45 


78 


56  45  4  0  6 

0.000  +  4  6  2  9  2  5 

1  7  9  5  78  1 

-  1  0  6  8 

9  1  6  3  0 

-  2  7  0  4 

-3*500  +  89  8  7 

+ - + - + - + - + - + 

10  20  30  40  50 


Plot  of  sixed  second  difference  of  log  GNPs 
Vertloel  eoele,  labeled  '1000D1D4'.  ie  Z(T).  where 
Z(T)  ■  1000C (Y(T)-Y(T-I ) )  -  (7(T-4)-Y(T-5))J  and 
Y(T)  >  log  GNP  (base  10) \  Tel  la  1947-2,  T«2  le 
1947-3.  ....  T*9  Is  1949-2,  «to*  (N«50) 


A  first  dlfferenoe  corresponds  to  a  veloolty.  The  dlfferenoe 

y(t)  -  y(t-1)  «  d(t), 

say,  Is  Cy( t)  -  y( t-1 ) ]/Ct-(t-1 ) ] ,  whioh  is  the  ohange  in  y  over  the 
one  tine  unit  fron  tine  t-1  to  tlae  t.  A  seoond  dlfferenoe 

d(t)  -  d(t-1) , 

whioh  in  teres  of  an  original  series  of  y’s  is 

{[y(t)-y(t-1)]  -  Cy(t-1)-y(t-2)]}, 

is  proportional  to  the  ohange  In  veloolty  aoross  the  indloated  tlae 
periods  and  henoe  is  essentially  an  aooeleratlon.  Thus  a  seoond 
dlfferenoe  is  perhaps  a  natural  transform  to  analyse.  (Note,  however, 
that  the  seoond  difference  used  here  Is  a  elxed  seoond  dlfferenoe, 
naaely,  the  ordinary  dlfferenoe  of  the  lag- four  dlfferenoe 
y(t)-y(t-4).]  Ths  seoond-dlfferenoe  series  appears  level.  Mhat  I  think 
Is  particularly  Interesting  is  that  the  aooeleratlon  of  the  eoonoay  due 
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to  the  Korean  conflict  of  the  1950’s  Is  readily  apparent.  (T  =  12  is 
1950-1,  T  s  13  is  1950-2,  eto.)  I  think  that  the  need  for  segmentation 
Is  olearly  lndioated.  One  needs  to  give  some  sort  of  speoial  treatment 
to  those  four  exceedingly  high  values. 

An  alternative  approach  to  such  observations  is  to  identify  then  as 
"outliers."  However,  "outlier"  connotes  spuriousness.  If  outlying 
observations  are  non-spurious  or  are  assooiated  with  a  reourring  oause, 
perhaps  they  should  not  then  be  termed  "outliers."  They  should  be 
modeled. 

In  other  oases,  where,  e.g.,  equipment  failure  is  suspected,  one 
truly  wants  to  look  for  outliers.  Here,  boo,  segmentation  oan  be 
useful. 

De  Alba  and  Zartman  (1979)  analyzed  radiobelemebrlo  measurements  of 
oows'  temperatures,  in  order  to  looate  the  periods  of  high  estrus,  with 
a  view  toward  more  optimally  timed  breeding  and  efficient  milk 
production.  A  technique  of  de  Alba  and  Van  Ryzin  (1979,  I960)  was 
used.  In  the  de  Alba-Zartman  report  the  analysis  of  the  temperatures  of 
one  cow  over  133  days  is  diaoussed  in  detail.  Eleven  observations  were 
deteoted  as  coming  from  distributions  with  means  shifted  relative  to  the 
rest  of  the  observations.  Inspection  of  the  data  showed  that  some  of 
these  temperatures  were  high  and  some  extremely  low.  It  is  concluded 
that  the  high  temperature  readings  correspond  to  times  of  motive  estrus. 
(Perhaps  the  low  readings  correspond  to  instrument  failure  and  hence  are 
true  "outliers"  in  that  sense.) 

Here,  as  in  de  Alba  and  Zartman  (1979),  the  raw  data  were 
pre-processed  by  differenoing.  Model-selection  orlterla  (see  below) 
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estimated  the  number,  k,  of  classes  of  segment  as  two,  but  k  =  3  and 
k  a  6  soored  almost  as  well.  Since  the  numbers  analyzed  were 
differences,  it  is  perhaps  especially  Interesting  to  consider  three 
olasses,  particularly  sinoe  the  segmentation  yielded  one  olass  for 
positive  differences,  one  for  negative  differences,  and  one  for 
differences  olnse  to  zero.  The  results  obtained  are  similar  to  those 
obtained  by  de  Alba  and  Zartman.  They  identified  eleven  observations  as 
"outliers;11  7  of  these  were  high  and  4  were  low.  The  segmentation 
algorithm  run  with  three  olasses  of  segment  identified  4  observations  as 
high;  these  were  among  the  7  identified  by  de  Alba  and  Zartman.  Use  of 
six  olasses  provided  even  oloser  agreement  to  the  de  Alba-Zartman 
results.  The  upper  two  of  the  six  classes  oaptured  6  of  the  7  high 
observations;  the  4  low  observations  were  looated  by  the  lowest  olass. 

(In  dlsoussion  after  the  paper  Professor  Parson  mentioned  that  at 
Texas  A&M  University  also  they  had  dealt  with  the  analysis  of  bovine 
estrus  and  found  by  spectral  analysis  that  a  filtered  Poisson 
process  —  see,  o.g.,  Parmen  (1962)  —  provided  a  good  fit  to  the  data.) 

Now,  having  dlaoussed  some  examples,  let  me  be  a  little  more 
speoiflo  about  tho  model  and  the  algorithm.  [More  formal  presentations 
are  found  in  Solove  ( 1983a, c,d).]  The  elements  of  the  segmentation 
model  are  the  olass- conditional  time  series  or  distributions,  with  their 
parameters;  the  labels;  and  the  transition  probabilities  between  the 
labels.  Correspondingly,  the  algorithm  alternates  between  estimation  of 
the  distributional  parameters,  estimation  of  the  labels,  and  estimation 
of  the  transition  probabilities.  That  is,  given  a  tentative  labeling, 
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one  can  obtain  tentative  eatimatea  of  the  parametera  of  the 
olass-oondltional  distributions  and  of  the  transition  probabilities. 
One  then  relabels  the  observations,  using  theae  updated  parameter 
estimates.  The  relabeling  is  done  as  follows:  If  x(t)  (i»e«,  if  tine 
period  t)  is  ourrently  labeled  as  olass  o,  then  x(t+1)  la  labeled  aa 
that  olass  d  for  whioh  the  produot 

p(o  to  d)f(x(t+1) Id) 

la  aaxinal,  where  p(o  to  d)  denotes  the  ourrent  estlnate  of  the 
probability  of  a  transition  fron  o  to  d  and  f(x|d)  denotes  the 
tentatively  estimated  olaas-d  probability  density,  evaluated  at  x. 
This  makes  sense  beoause,  under  the  assumptions  of  the  model,  the 
likelihood  is  the  produot  of  these  terns* 

To  Illustrate  the  algorithm,  let  us  oonslder  a  short,  artificial 
time  series. 

t:  1  2  3  4  5  6  ?  8  9  10  11  12 

x(t):  113121267111 

Suppose  it  is  specified  that  there  are  two  olasses  and  that  the 
olaas>oondltlonal  distributions  are  exponential.  Suppose  the  initial 
guesses  of  the  parameters  ara  equal  prior  probabilities  for  the  two 
olasses  and  means  of  2  and  3.  Then  initially  the  olasa-oondltional 
densities  are  taken  as  f(xla)  ■  (1/2)exp(-x/2)  and  f(xlb) 
■  (1/3)exp(-x/3).  In  the  first  Iteration,  using  equal  prior 

probabilities  of  .5,  one  labels  x  as  having  oome  fron  Class  A  if 
.5f(x!a)  >  .5f(x|b),  whioh  simplifies  to  x  <  2.43.  This  gives  the 
following  estimated  labels  at  the  end  of  the  first  iteration. 
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t :  1  2  3  4  5  6  7  8  9  10  It  12 

*(t):  1  13121287111 

labels  aabaaaabbaaa 

Non  tha  transition  probabilities  oan  ba  estinated  froa  th«  aaquanca  of 
astlaabad  labels.  Tha  11  transitions  ares  a  to  a,  a  to  b,  b  to  a, 
a  bo  a,  a  bo  a,  a  bo  a,  a  to  b,  b  to  b,  b  to  a,  a  to  a,  a  to  a.  These 
give  tha  following  esbinabes  of  the  transition  probabilities. 

p(a  bo  a)  *  3/4  p(a  to  b)  *  1/4 

p(b  to  a)  «  2/3  p(b  to  b)  ■  1/3 

Tha  olass  naans  are  estinated  as  follows  (but  sea  Seotlpn  6t  Plans  for 
* 

Further  Research,  for  a  disoussion  of  this  point). 

Mean  of  a'ss  (1  +  1  +  1+  2+1+2+1  +  1  +  1)/9  ■  11/9  *  1.22 
Mean  of  b'si  (3  +  6  +  7)/3  -  16/3  “  5.33 
Now  the  oondltlon  for  labeling  the  current  x  as  NaN,  given  that  the 
preceding  x  has  been  labeled  as  Han,  is 

p(a  to  b)f(x|b)  <  p(a  to  a)f(xla), 
or 

(1/4)(1/5.33)«xp(-x/5.33)  <  <3/4)(1/1.22)exp(-x/1.22), 
which  slnpliries  to  x  <  4.075.  Slnllarly,  the  oondltlon  for  labeling  x 
as  "a",  given  that  the  preoeding  observation  has  been  labeled  as  "b",  is 
p(b  to  b)f(xlb)  <  p(b  to  a)f(xla),  whloh  slnplifies  to  x  <  1.24. 
These  seoond- iteration  classification  rules  ohange  only  the  label  of 
x(3)  fron  "bM  to  "a". 

At  this  point  let  ne  nention  a  possible  lnproveaent  to  the 
algorltha.  At  the  relabeling  stage,  where  the  labels  are  re-esblaabed, 
based  on  tentative  paraneter  estinabes  and  transition  probabilities,  the 
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problem  la  one  of  estimating  a  finite  sequence,  given  the  transition 
probabilities.  The  dynamic  programming  approaoh  to  this  problem  la 
Known  as  the  Viterbl  algorithm.  [Forney  (1971)  is  perhaps  the  most 
readable  reference  on  this;  the  references  Viterbl  (1967#  1971)  end 
Viterbl  and  Odenwalder  (1969)  ere  also  given.)  This  approaoh  will  be 
applied  in  the  future.  The  present  approach  is  perhaps  somewhat  simpler 
but  has  its  advantages.  It  adapts  Itself  to  operation  in  the  purely 
sequential  mode,  and  it  is  relatively  easy  to  program. 

An  algorithm,  such  as  the  present  one,  whioh  alternates  between 
optimizing  different  sets  of  variables,  is  known  as  a  "relaxation” 
method  (Southwell  1940,  1946;  Ortega  and  Rhainboldt  1970).  In  the 
present  case  the  different  sets  of  variables  are  different  parameters, 
namely,  the  distributional  parameters,  the  transition  probabilities,  and 
the  labels. 

We  note  in  passing  that  the  EH  algorithm  —  see  Dempster,  Laird  and 
Rubin  (1977)  —  is  auoh  a  relaxation  method,  where  at  the  "E"  step  the 
estimation  Is  by  expectation  and  at  the  "H"  step  the  estimation  is  by 
maximization. 

An  alternative  approaoh  to  the  presently-implemented  relaxation 
method  would  involve  equating  partial  derivatives  of  the  likelihood 
function  to  aero  and  solving  the  resulting  equations.  It  is  clear  that 
results  analogous  to  those  of  Wolfe  (1970)  will  be  obtained  by  this 
approaoh,  with  transition  probabilities  replacing  his  mixture 
probabilities.  In  any  case,  the  resulting  equations  have  to  be  solved 
by  an  iterative,  numerical  method,  and  it  is  not  dear  whether  it  would 
be  any  better  than  the  the  presently-implemented  relaxation  method. 
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4.  Imags-aegmentation  experiments 

Conventional  approaches  bo  segmentation  (see,  e.g.,  Ahuja  and 
Rosenfeld  (1982)  for  a  survey  of  iaage  models)  inolude  ordinary 
clustering;  edge  deteotors  suoh  aa  the  two-dimensional  filters  of 
Irwin  Sobel  or  Judy  Prewitt;  and  a  pixel-labeling  method  that  has 
(perhaps  inaoourately)  been  called  a  relaxation  method.  This  Involves 
updating  the  ourrenb  estimate  of 

Pr( label  of  t  is  o I  data), 

the  (conditional)  probability  that  the  true  label  of  pixel  b  is  olass  o, 
given  the  data,  by  an  updated  version  of  these  estimated  probabilities, 
where  the  updating  takes  into  aooount  the  ourrent  labels  of  neighboring 
pixels.  This  Is  done  by  means  of  "compatibility  coefficients," 
measuring  the  consistency  of  label  o  for  pixel  t  with  the  ourrent 
labelings  of  neighboring  pixels.  (See,  e.g.,  Eklundh,  Yamamoto  and 
Rosenfeld  1980.)  More  precisely,  to  estimate  Pr(label  of  t  is  oldata), 
one  moves  from  stage  s-1  to  stage  a  as  follows.  Let 

Pr(label  of  t  is  oidata;s) 
be  the  s-th  stage  estimate.  Then 

Pr(label  of  t  Is  o|data;s)  « 

Pr(label  of  t  is  o)ldata;s-1)  C(t,o;s-1 )/C, 
where  C  is  a  normalising  faotor  equal  to  the  sum  of  the  denominators 
In  this  expression  and  C(t,o;s-1)  la  the  compatibility  coefficient; 
e.g.,  C(t,o;s-1)  oould  be  related  to  estimated  transition  probabilities 
for  the  labels  or  taken  aa  the  proportion  of  the  neighbors  of  pixel  t 
which  are  labeled  o  at  stage  s-1.  A  problem  with  this  relaxation 
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method,  noted  by  various  researchers,  is  that  the  image  gets  good,  then 
"fades,"  so  there  Is  a  problem  of  knowing  when  to  stop  the  iteration. 

nils  approaoh  is  Intuitively  attractive.  The  approaoh  I  use  seems 
to  have  the  same  intuitive  attraction,  plus  the  advantage  of  being 
embedded  in  a  Markov  model  whtoh  gives  a  likelihood  function,  putting 
the  problem  in  the  oontext  of  parameter  estimation  in  a  probability 
model  and  making  performance  evaluation  possible.  (For  instanoe,  one 
oan  consider  asking  questions  suoh  as,  "What  if  we  worked  with 
first-order  neighbors  and  the  model  were  really  seoond-order?"  I'm  not 
saying  that  suoh  questions  are  easy  to  answer;  I'm  merely  stating  that 
it  is  only  In  the  presence  of  a  model  that  their  formulation  is  even 
possible.) 

The  idea  of  the  model  in  the  oase  of  images  is  the  same  as  for  time 
ssrlQs,  but  the  transition  matrix  is  more  oomplloated.  The  transition 
probabilities  are,  even  in  the  simplest,  first-order,  one-sided  oase 
(where  one  conditions  only  on  pixels  to  the  north  and  west  of  the  given 
pixel,  rather  than  those  to  the  north,  west,  south,  and  east),  functions 
p((o,d)  to  e)  of  three  arguments,  where  this  represents  the  probability 
of  a  transition  to  class  e  in  pixel  t,  given  that  the  pixel  to  the  north 
of  t  is  olass  c  and  the  pixel  to  the  west  of  t  is  olass  d.  [The  Markov 
approaoh  used  here  is  not  unrelated  to  that  presented  by  Professor 
Grenander  at  this  confer enoe  (Grenander  1985;  see  also  Grenander 
1983).]  The  segmentation  algorithm  disoussed  in  the  present  paper,  like 
the  "relaxation"  method  using  compatibility  ooeffloients,  also  has  the 
property  that  eaoh  iteration  is  not  neoessarily  better  than  the 
preoeding,  but  there  is  a  likelihood  value  associated  with  eaoh 
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iterabion,  and  these  values  oan  be  used  to  piok  out  the  best  iteration. 

Here  are  results  of  sons  experinents  with  the  segmentation 

algorithm. 

The  Fisher  iris  data  oonslst  of  4  variables  observed  for  each  of 
150  irises,  50  in  eaoh  of  three  species.  In  order  to  form  an 
experimental  image,  these  150  were  arranged  into  15  rows  of  10,  the 
first  5  rows  being  speoies  1,  rows  6-10  being  speoies  2,  and  rows  11-15 
being  speoies  3*  Thus  the  true  segmentation  looks  like  this. 


1111111111 

1111111111 

1111111111 

1111111111 

1111111111 

2222222222 

2222222222 

2222222222 

2222222222 

2222222222 

3333333333 

3333333333 

3333333333 

3333333333 

3333333333 


The  measurements  on  flowers  50,  100,  and  150  were  used  as  initial 
estimates  for  the  distributional  parameters,  namely,  means  of 
multivariate  normal  distributions.  A  oommon  covarlanoe  matrix  was 
assumed.  (Aotually,  statistical  tests  and  model-selection  criteria 
suggest  that  different  cover lance  matrioes  should  be  used;  my  algorithms 
do  allow  for  this.)  The  initial  estimate  of  the  oommon  covariance  was 
taken  to  be  proportional  to  the  identity  matrix.  Equal  prior 
probabilities  for  the  three  olassea  were  assumed  as  Initial  estimates. 
After  only  three  iterations,  no  label  ohanged.  The  segmentation 
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obtained  was  as  follows. 


1111111111 

1111111111 

1111111111 

1111111111 

1111111111 

2323232323 

2222222222 

3222222222 

2222222222 

2222222222 

3333333333 

3333333333 

3333333333 

3333333333 

3333333333 


Experiments  wars  run  with  different  configurations  of  these  data. 
Below  is  the  true  configuration  for  Experiaent  1.  The  four-by-four 
block  of  2's  in  the  aiddle  oan  be  thought  of  as  a  target  to  be  looated. 


1111111111 

1111111111 

1111111111 

1112222111 

1112222111 

3332222333 

3332222333 

3333333333 

3333333333 

3333333333 


The  segnentation  produoed  was  as  follows. 

1111111111 

1111111111 

1111111111 

1113232111 

1113222111 

3332222333 

2332222333 

3333333333 

3333333333 

3333333333 
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The  following  was  the  true  configuration  for  Experiment  2.  Note 

the  saaU  "target"  of  four  2* a. 

1111111111 
1111111111 
1111111111 
1111111111 
11112  2  1111 
3333223333 
3333333333 
3333333333 
3333333333 
3333333333 


The  segmentation  obtained  was  as  follows. 


1111111111 

1111111111 

1111111111 

1111111111 

1111321111 

3333323323 

3333333333 

3333333333 

3333333333 

3333333333 


Note  the  algorithm  worked  reasonably  well  in  deteotlng  even  a  small 
target,  even  with  the  same  initial  (equal)  estimates  of  prior 
probabilities,  quite  different  from  the  true  probabilities  of  .48,  .04, 
.48  for  the  three  olasses.  Thus  perhaps  the  algorithm  will  not  have  to 
be  well  "tuned"  in  order  to  work  well. 

Note  there  are  problems  at  the  northwest  edges,  due  to  the  way  the 
algorithm  is  set  up.  In  future  software  development  I  might  program  a 
prooedure  analogous  to  Box  and  Jenkins'  baok- forecasting  ("baokoasting") 
to  take  oare  of  this.  Then,  after  a  first  segmentation,  the  data  would 
be  read  through  in  reverse  order,  so  that  northwest  pixels  beooae 
southeast  pixels,  easy  to  label  oorreotly. 
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5.  Model-selection  criteria 

A  question  of  obvious  importance  is  that  of  how  many  desses  of 

segment  to  use,  that  is,  what  should  be  the  value  of  the  parameter  k? 

Maximum  likelihood  estimation  alone  cannot  provide  an  answer,  because  it 

is  model-conditional;  that  is,  maximum  likelihood  applies  to  the  problem 

of  estimating  the  other  parameters,  for  a  fixed  value  of  k.  It  does  not 

apply  bo  the  problem  of  estimating  k  because  the  likelihood  Itself 

ohanges  as  k  does.  An  approach  to  the  problem  of  estimation  of  k  is 

provided  by  model-selection  criteria,  such  as  Akaike's,  Schwarz'  and 

Kashyap's.  These  criteria  add  a  penalty  for  using  extra  parameters  bo 

the  negative  log-likelihood  funotion.  Thus  a  small  score  is  a  good 

score  on  these  criteria.  Akaike's  orlterlon  is  based  on  a  heurlstlo 

estimate  of  the  oross-entropy  of  the  true  model  and  the  model  with  k 

classes.  (See  Parzen,  1982.)  Schwarz'  orlterlon,  perhaps  more 

convincing,  is  based  on  a  Bayesian  approaoh.  Kashyap  obtains  Sohwarz' 

criterion  by  a  relatively  simple  expansion,  then  takes  this  expansion  a 

term  further  to  obtain  another  criterion,  which  I  call  Kashyap's 

criterion.  The  additional  term  seems  to  be  particularly  meaningful. 

These  model-selection  oriterla  take  the  form 

-2  log  L(k)  +  a(n)m(k)  +  b(k), 

where  "log"  here  denotes  the  natural  (base-e)  logarithm,  and 

L(k)  s  likelihood  under  the  k-th  model,  maximized 

with  respect  bo  the  parameters, 

a(n)  s  2,  for  all  n,  b(k)  =  0,  for  Akaike's  criterion, 

a(n)  s  log  n,  b(k)  a  0,  for  Schwarz'  orlterlon, 

a(n)  log  n,  b(k)  s  log[det  B(k),  for  Kashyap's  orlterlon, 

det  =  determinant, 
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and 

B(k)  s  aatrix  of  second  parbial  derivatives  of  log  L(k) 
with  respeob  bo  the  parameters. 

(The  sathemationl  expectation  of  B(k)  is  the  Fisher  information 
matrix.)  The  optimal  k  is  that  value  which  minimizes  the  criterion. 
Akaike's  criterion  generally  ohcoras  a  higher  value  of  k  (more 
parameters)  than  do  the  others.  Sinoe  for  n  greater  than  8,  log  n  is 
greater  than  2,  Schwarz'  criterion  will  ohoose  a  value  of  k  no  larger 
than  that  chosen  by  Akaike's,  for  n  grantor  than  8. 

6.  Plans  for  further  research 

Several  items  for  future  research  have  already  been  mentioned, 
including  programming  of  backoasting  and  use  of  the  Viberbi  algorithm. 

Some  other  plans  for  additional  researoh  will  be  mentioned  now. 
First  there  is  the  matter  of  improved  estimation  of  the  distributional 
parameters.  For  purposes  of  dlaousslon  foous  on  the  example  of 
segmenting  a  time  series  into  two  olasses,  i.e.,  labeling  eaoh 
observation  as  an  "aM  or  a  "b."  There  is  a  truncation  resulting  from 
the  present  implementation  of  the  algorithm.  Namely,  only  those 
observations  labeled  "a"  will  be  used  in  updating  the  ourrenb  estimate 
of  the  mean  of  Class  A.  But  these  observations  are  a  truncated  sample 
from  Distribution  A,  and  the  algorithm  does  not  treat  them  as  suoh.  (At 
the  Conferenoe,  Professor  Tukey  vary  kindly  sought  me  out  the  day  after 
my  presentation  to  point  this  out  to  me,  and  also  to  remark  on  the 
limitations  of  the  unidirectional  approach  of  the  algorithm;  see  below.) 
Rather  than  deal  with  the  truncation  per  se,  I  had  planned  in  the  next 
stage  of  the  work  to  modify  the  estimators  by  doing  them  Bayeaianly, 
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e.g.,  estimate  the  mean  of  Group  A  as  N/D,  where 

N  s  x(l)  Pr(a|x(1))  +  x(2)  Pr(a|x(2))  +  ...  +  x(n>  Pr(alx(n)) 
and 

D  a  [Pr(a|x(1))  +  Pr(a|x(2))  ...  +  Pr(a|x(n))] 

(In  this  expression,  Pr(alx)  can  be  replaced  by  Pr(x!a)  sinoe  Pr(alx)  a 
f(xla)Pr(a)/f(x)  and  Pr(a)/f(x)  is  a  common  factor  which  will  oanoel 
out.)  In  this  estimate,  all  the  observations  play  a  role,  whether 
labeled  as  "a"  or  "b»"  so  that  at  least  some  of  the  bias  will  be  removed 
by  allowing  the  larger  "b"  observations  to  enter. 

I  obtain  the  likelihood  function  by  a  one-sided  approaoh, 
conditioning  any  given  pixel  on  the  results  in  the  pixels  to  its  north 
and  west.  A  two-sided,  full  neighborhood  approaoh  seems  preferable  to  a 
unidirectional  one.  The  unidirectional  approaoh  is  a  devioe  for  writing 
down  the  likelihood,  but  this  does  not  mean  one  has  to  be  wedded  to  that 
approach  in  the  iterative  updating.  That  is,  the  parameters  oan  be 
estimated  with  a  full  neighborhood  approaoh. 

Another  bit  of  further  researoh  is  to  oaloulate  Kashyap's  criterion 
for  various  clustering  and  segmentation  models.  Also,  so  far  the 
method,  algorithm  and  software  have  been  developed  only  for  the  oase 
where  the  observations  within  a  class  are  independent  (and  Gaussian).  A 
next  step  will  be  autoregression  within  classes.  This  is  of  obvious 
importance  in  time  series,  and  in  the  context  of  images,  it  oan  be  used 
to  model  textures.  Still  another  generalization  is  to  allow  some  forms 
of  time  or  state  dependency  in  the  transition  probabilities. 
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RECENT  RESEARCH  IN  EXPERIMENTAL  DESICN  FOR  QUALITY  IMPROVEMENT 
WITH  APPLICATIONS  TO  LOGISTICS 


George  E.  P.  Box 


1.  LOGISTICS  AND  QUALITY  CONTROL 

A  traditional  philosophy  of  quality  control  has  been  to  "inspect  bad 
quality  out"  and  indeed  there  are  famous  military  standards  that  employ  this 
philosophy.  W.  Edwards  Darning  (1982)  has  likened  this  to  making  toast 
according  to  the  recipe  "you  burn  it  and  I'll  scrape  it",  and  has  urged  tha 
alternative  philosophy  of  assuring  that  good  quality  has  been  built  In  to  the 
product  in  the  first  place.  In  particular  he  attributes  to  the  latter 
philosophy  the  success  of  Japanese  industry  in  producing  high  quality  products 
at  low  cost.  A  typical  axampla  of  tha  dramatic  consequences  that  have  been 
attributed  to  these  differences  of  approach  are  the  air-conditioner  defect 
rates  shown  in  Table  1  end  quoted  by  David  Garvin  (1983). 


(In  tha  factory t  Assembly  line  defects  per  100  units) 

American  Japaneaa 


Total  .. .  63.5  0.95 

Leaks  . 3.1  0.12 

Electrical  .  3.3  0.12 


(In  tha  flaldi  Sarvica  call  rate  per  100  units  undar 
first  year  warranty  coverage) 


American  Japanese 

Total  . .  10.5  0.6 

Compressors  . . . .  1.0  0.05 

Tharmostats  . * . .  1.4  0.002 

Fan  motors  . •***••*•  0.5  0.028 


TABLE  1.  Defect  rates  in  U3  and  Japanese  air  conditioners 


Similar  comparisons  have  bean  mads  between  defect  rates  in  American  and 
Japanese  automobiles. 

The  same  United  States  industry  that  makes  air  conditioners  and  motor 
vehicles  also  makes  military  hardware.  It  seems  clear  thsrsfors  that  a  major 
change  in  quality  philosophy  could  produce  s  major  improvement  in  the 
reliability  of  the  Army's  equipment.  The  philosophy  of  "building  quality  in" 
employs  a  policy  of  never  ending  quality  improvement  which  may  be  typified  in 
terms  of  the  traditional  statistical  model 


I 


where  y  ia  *  quality  characteristic  believed  to  depend  on  a  set  of  variables 
denoted  by  whose  identity  ie  known,  and  e  is  the  difference  y  -  f(£^) 

usually  referred  to  as  error.  (Suc,h  "errors"  are  often  somewhat  arbitrarily 
imbued  by  the  theoretician  with  properties  of  randomness,  normality 
independence  and  homoscedastioity) .  In  reality  e  is  a  function  e(;g,,)  of  a 
number  of  additional  variables,  x_  say,  which  affect  the  system  but  whose 
identity  is  usually  unknown.  In  general,  quality  improvement  is  achieved  by 
transferring  elements  of  the  unknown  factor  vector  Into  the  known  factor 
vector  x4  as  indicated  below 

’*  i 

y  -  *  •hs2) 

known  unknown 

The  effect  of  such  transfer  la  two-fold 

(i)  to  reveal  effects  of  previously  unknown  factors  which  may  then  be 

adjusted  to  levels  yielding  higher  quality  and/or  used  to  control  the 
process. 

(ii)  to  remove  variation  previously  caused  by  haphssard  changes  in  these 
factors. 

Some  of  the  statistical  techniques  which  contribute  to  this  transfer  are 
quality  control  charting  (including  Shewhart,  Cusum,  Pareto  and  Fishbone 
charts)  and  designed  experimentation  on  line  and  off  line  (employing  in 
different  and  appropriate  contexts  factorial,  fractional  factorial  and 
orthogonal  array  designs,  evolutionary  operation  and  response  surf see 
methods) . 


2.  SCIENTIFIC  METHOD  AND  QUALITY 

Charting  and  experimentation  are  examples  respectively  of  passive 
surveillance  and  active  Intervention  both  of  which  are  important  elements  in 
scientific  method  which  it  is  desirable  to  consider  further. 

Humana  differ  from  other  aniuala  most  remarkably  in  their  ability  to 
learn.  It  is  clear  that  although  throughout  the  history  of  mankind 
technological  learning  has  taken  place,  until  three  or  four  hundred  years  ago 
change  occurred  very  slowly.  One  reason  for  this  was  that  in  order  to  learn 
something  -  for  example,  how  to  make  fire  or  champagne  -  two  rare  events 
needed  to  coinoidsi  (a)  an  informative  event  had  to  occur,  and  (b)  a  person 
able  to  draw  logical  conclusions  and  to  act  on  them  had  to  be  aware,  of  that 
informative  event. 

Passive  surveillance  is  a  way  of  increasing  the  probability  that  the  rare 
informative  event  will  be  constructively  taken  note  of  and  is  exemplified  by 
quality  charting  methods.  Thus  a  Shswhart  chart  is  a  means  to  ensure  that 


possibly  informative  events  are  brought  to  the  attention  of  those  who  may  be 
able  to  discover  in  them  an  "assignable  cause"  (Shewhart  1931)  and  act 
appropriately. 


Active  intervention  by  experimentation  aims,  in  addition,  to  increase  the 
probability  of  an  informative  event  actually  occurring.  A  designed  experiment 
conducted  by  a  qualified  experimenter  can  dramatically  increase  the 
probability  of  learning  because  it  increases  simultaneously  the  probability  of 
an  informative  ev&nt  occurring  and  also  the  probability  of  the  event  being 
constructively  witnessed.  Recently  there  has  been  much  use  olr  experimental 
design  in  Japanese  industry  particularly  by  Genichi  Taguchi  (Taguchi  and  Wu 
(I960))  and  his  followers.  In  off-line  experimentation  he  has  in  particular 
emphasized  the  use  of  highly  fractionated  designs  and  orthogonal  arrays  and 
the  minimization  of  variance. 

In  the  remainder  of  this  paper  we  briefly  outline  some  recent  research  on 
the  use  of  experimental  design  in  the  improvement  of  quality. 


3.  USE  Of  SCREENING  DESIGNS  TO  IMPROVE  QUALITY 

Table  2  shows  in  summary  a  highly  fractionated  two-level  factorial  design 
employed  as  s  screening  design  in  an  off-line  welding  experiment  performed  by 
the  National  Railway  Corporation  of  Japan  (Taguchi  and  wu,  1980).  In  the 
column  to  the  right  of  the  table  is  shown  ths  observed  tensile  strength  of  the 
weld,  one  of  several  quality  characteristics  measured. 

The  design  wen  chosen  on  ths  assumption,  tha*  in  addition  to  main  effects 
only  th*  two-factor  interactions  AC,  AG,  AH,  and  GH  wsrs  expected  to  be 
present.  On  that  supposition,  all  nine  main  effects  end  the  four  selected 
two-factor  interactions  can  be  separately  estimated  by  appropriate  orthogonal 
contrasts,  ths  two  remaining  contrasts  corresponding  to  the  columns  labelled 
e^  and  «2  measure  only  experimental  error.  Below  the  table  are  shown  the 
grand  average,  the  fifteen  effect  contrasts,  and  the  effects  plotted  on  a  dot 
diagram.  When  the  effects  are  plotted  on  normal  probability  paper,  thirteen 
of  them  plot  roughly  as  a  straight  line  but  the  remaining  two,  corresponding 
to  the  main  effects  for  factors  B  and  C,  fall  markedly  off  the  line, 
suggesting  that  over  the  ranges  studied,  only  factors  S  and  C  affect 
tensile  location  by  amounts  not  readily  attributed  to  noise. 

If  this  conjecture  is  true,  then,  st  least  approximately,  the  sixteen 
runs  could  bo  regarded  as  four  replications  of  a  22  factorial  design  in 
factors  B  and  C  only*  However,  when  ths  results  are  plotted  in  Figure  1 
so  av  to  reflect  this,  inspection  suggests  the  existence  of  a  dramatic  effect 
of  a  different  kind  -  when  factor  C  is  at  its  plus  lsvel  ths  spread  of  the 


*To  facilitate  later  discussion  we  have  set  out  the  design  and  labelled  the 
levels  somewhat  differently  from  Taguchi. 


At  Kind  of  Balding  Bods 
E:  Period  of  Drying 
C:  fielded  Material 
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TABLE  2.  A  fractional  two-level  design  used  in  a  welding  experiment  shoving  observed  tensile 
strength  and  effects.  Below  the  estimated  effects  are  plotted  as  a  dot  diagram. 


data  appears  much  larger  than  when  It  Is  at  its  minus  level.  Thus,  in 
addition  to  detecting  shifts  in  location  due  to  B  and  C,  the  experiment 
may  also  have  detected  what  we  will  call  a  dispersion  effect,  due  to  C.  The 
example  raises  the  general  possibility  of  analysing  unreplicated  designs  for 
dispersion  effects  aa  well  as  for  the  more  usual  location  effects. 


OODA 


Figure  1. 


Drying  Period  8 


Tensile  data  as  four  replicates  of  a 
design  in  factors  B  and  c  only. 


factorial 


Date  of  this  kind  might  be  accounted  for  by  the  effect  of  one  or  more 
variables  other  than  B  that  affected  tensile  strength  only  at  the  "plus 
level"  of  C  (only  when  the  alternative  material  was  used).  Analysis  of  the 
eight  runs  made  et  the  plus  level  of  C  does  not  support  this  possibility, 
however . 
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RATIONALES  FOR  USING  SCREENING  DESIGNS 


Before  proceeding  we  need  to  consider  the  question,  Hln  what  situations 
are  screening  designs,  such  as  highly  fractionated  factorials,  useful?" 

4.1.  Effect  Sparsity 

A  common  industrial  problem  is  to  find  from  a  rather  large  number  of 
factors  those  few  that  are  responsible  for  large  effects.  The  idaa  is 
comparable  to  that  which  motivates  the  use  in  quality  control  studies  of  the 
"Pareto  diagram."  (See,  for  example,  Zshikawa  (1976)).  The  situation  is 
approximated  by  postulating  that  only  a  small  proportion  of  effects  will  be 
"active"  and  the  rest  "inert".  We  call  this  the  postulate  of  effect 
sparsity.  For  studying  such  situations,  higly  fractionated  designs  and  other 
orthogonal  arrays  (Finney  (1945),  Plackett  and  Buraan  (1946),  Rao  (1947), 
Taguchi  and  Wu  (1980))  which  can  screen  moderately  large  numbers  of  variables 
in  rather  few  runs  are  of  great  interest.  Two  main  rationalisations  have  been 
suggested  for  the  use  of  these  designs}  both  ideas  rely  on  the  postulate  of 
effect  sparsity  but  in  somewhat  different  ways. 

4.2.  Rationale  Based  on  Prior  Selection  of  Important  Interactions 

Zt  is  argued  (see  for  example  Davies  (1954))  that  in  some  circumstances 
physical  knowledge  of  the  process  will  make  only  a -few  interactions  likely  and 
that  the  remainder  may  be  assumed  negligible.  For  example,  in  the  welding 
experiment  described  above  there  were  36  possible  two-factor  interactions 
between  the  nine  factors,  but  only  four  were  regarded  as  likely,  leaving  32 
such  interactions  assumed  negligible.  The  difficulty  with  this  idea  is  that 
in  many  applications  the  picking  out  of  a  few  "likely"  Interactions  is 
difficult  if  not  impossible.  Indeed  the  investigator  might  justifiably 
protest  thajt,  in  the  circumstance  where  an  experiment  is  needed  to  determine 
which  first  order  (main)  effeots  are  Important,  it  is  illogical  that  he  be 
expected  to  guess  in  advance  which  effects  of  second  order  (interactions)  are 
important . 

4.3.  Projective  Rationale  Factor  8parslty 

A  somewhat  different  notion  is  that  of  factor  sparsity.  Thus  suppose 
that,  of  the  k  factors  considered,  only  a  small  subset  of  unknown  size  d, 
whose  identity  is  however  unknown,  will  be  active  in  providing  main  effects 
and  interactions  within  that  subset.  Arguing  as  in  Box  and  Hunter  (1961)  a 
two-level  design  enabling  us  to  study  such  a  system  is  a  fraction  of 
resolution  R  ■  d  +  1  (or  in  the  terminology  of  Rao  (1947)  an  array  of 
itrength  d)  which,  produces  complete  factorials  (possibly  replicated)  in 
every  one  of  the  ( .;  spaces  of  d  ■  R  -  1  dimensions.  For  example,  we  have 
seen  that  on  the  assumption  that  only  factors  B  and  C  are  important,  the 
welding  design  could  be  regarded  as  four  replicates  of  a  22  factorial  in 
just  thosa  two  factors.  But  because  the  design  in  of  resolution  R  ■  3  the 
same  would  have  been  true  for  any  of  the  36  choices  of  two  out  of  the  nine 
factors  tested.  Thus  the  design  would  be  appropriate  if  it  were  believed  that 
not  more  than  two  of  the  factors  were  likely  to  be  "active" • 


1  2  3  4  5  6  7  8  9  10  11  12  13  14  15 
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TABLE  3.  Some  alternative  uses  of  the  orthogonal  array  of  Table  2. 


For  further  illustration  we  consider  again  the  sixtenn-run  orthogonal 
array  of  Table  2  and,  adopting  a  roman  subscript  to  denote  the  resolution  R 
of  the  design,  we  indicate  in  Table  3  various  ways  in  which  that  array  might 
be  used.  It  may  be  shown  that 

(a)  If  we  associated  the  fifteen  contrast  columns  of  the  design  with  1 

fifteen  factors,  we  would  generate  a  aifi1 1  providing  four-fold 

replication  of  2 2  factorials  in  every  one  of  the  105  two-dimensional 
projections, 

(b)  If  we  associated  only  columns  1,  2,  4,  7,  8,  11,  13,  and  14  with 
eight  factors  we  would  agenerate  a  2IV  design  providing  two-fold 
replication  of  23  factorials  In  every  one  of  the  56  three-dimensional 
projections, 

(o)  If  we  asft'jciated  only  columns  1,  2,  4,  U,  and  15  with  five  factors 
we  would  generate  a  2y  1  design  providing  «  24  factorial  in  every  one  of 
the  four-dimensional  projections, 

(d)  If  we  associated  only  columns  1,  2,  4,  and  8  with  four  factors  we 
would  obtain  the  complete  24  design  from  which  this  orthogonal  array  was  in 
fact  generated, 

Designs  (a),  (b)  and  (a)  would  thus  be  appropriate  for  situations  where  we 
believed  respectively  that  not  more  than  2,  3,  or  4  factors  would  be 
active  •  Notice  that  intermediate  values  of  k  could  be  accommodated  by 
suitably  omitting  certain  columns,  Thus  the  welding  design  is  a 
arrangement  which  can  be  obtained  by  omitting  6  columns  from  the  complete 
2III1  *  Notic#  finally  that  for  intermediate  designs  we  can  take  advantage  of 
both  rationales  by  arranging,  as  was  dona  for  the  welding  design,  that 
particular  interactions  are  isolated. 


Columns 

(a) 

,15-11 

•*111 

(b) 

2®“4 

2XV 

(c) 

25"1 

2V 

(d) 

V 

24 

*The  designs  give  partial  coverage  for  a  larger  number  of  factors,  for  example 

(Box  and  Hunter  (1961))  56  of  the  70  four-dimensional  projections  of  the  2®y4 
yield  a  full  factorial  in  four  variables,  A 


X  discussion  of  the  iterative  model  building  proceos  by  Box  and  Jenkins 
(1970)  characterized  three  steps  in  the  iterative  data  analysis  cycle 
indicated  below 


identification 


fitting 


diagnostic  checking 


Most  of  the  presont  paper  is  concerned  with  model  identification  -  the  search 
for  a  model  worthy  to  be  formally  entertained  and  fitted  by  an  efficient 
procedure  such  as  maximum  likelihood.  The  situation  we  now  address  concerns 
the  analysis  of  fractional  designs  such  as  the  welding  design  in  the  above 
context  when  only  a  few  of  the  factors  are  likely  to  have  effects  but  these 
may  include  dispersion  effects  as  well  as  location  effects. 


DISPERSION  EFFECTS 


He  again  use  the  design  of  Table  2  for  illustration.  There  are  16  runs 
from  which  16  quantities  —  the  average  and  IS  effect  contrasts  --  have  been 
calculated.  Now  if  we  were  also  interested  in  possible  dispersion  effects  we 
could  also  calculate  IS  variance  ratios.  For  example#  in  column  1  w«  can 
compute  the  sample  variance  a*m  for  those  observations  associated  with  a 
minus  sign  and  compare  it  with  the  sample  variance  e^+  for  observation* 
associated  with  a  plus  sign  to  provids  the  ratio  Ty  «  s^./s^.  IS  this  is 
done  for  the  welding  data  we  obtain  values  for  InF#*  given  in  Figure  2(e). 

It  will  be  recalled  that  in  the  earlier  analysis  a  large  dispersion  affect 
associated  with  factqr  C  (column  IS)  was  found# 'but  in  Figure  2\a)  the 
effect  for  factor  C  is  not  especially  extrama#  instaad  tha  dispersion  effect 
for  factor  d  (column  1)  stands  out  from  ell  the  rest.  This  misleading 
indication  occurs  because  w«  have  not  so  far  taken  account  of  the  aliasing  of 
location  and  dispersion  effects.  Since  sixtoen  linearly  independent  location 
effects  have  already  been  calculated  for  the  original  data#  calculated 
dispersion  effects  must  be  functions  of  these.  Recently  (Box  and  Meyer  1VB4u) 
e  general  theory  of  location-diaparsion  aliasing  has  been  obtained  for 
factorials  and  fractional  factorials  at  two  levels.  For  illustration#  in  this 
particular  example  it  turns  out  that  the  following  identity  exists  for  the 
dispersion  effect  F« #  that  in  tha  F  ratio  associated  with  factor  D  and 
hence  for  column  1  of  the  design. 

Aft*  AM.  A  A  A  A  A  A  A  A  m  A  A  A  A  A  . 

.  <a"3)a»<4-S)%(6»7)%(8-9)%(10-11)a!»(12-n)Vl14-15):..  # . , 

*4  *  A  A  A  A  A  n  A  A  A  A  A  a  *  A  A  A  A  m  A  A  *  *  *  * 

( 2+3 )  >  ( 4 +5 )  2+  ( 6+7 ) 2  +  ( 8 +9 )  >  ( 1 0 +11 )  U  ( 1 2 +1 3 )  *• +  ( 1 4+ 1 5 ) 4 

A  A  A  A 

Now  (see  Table  2)  14  «  B  -  2.15  and  15  »  C.  ■  3.10  are  the  two  largest 

location  effects#  standing  cut  from  all  the  othars.  The  extreme  value  of  Ft 
associated  with  an  apparent  dispersion  effect  of  factor  D(1)  is  largely 


*Xn  this  figure  familiar  normal  theory  eignif icance  lavels  are  also  shown. 
Obviously  the  necessary  assumptions  are  not  satisfied  in  this  case#  but  these 
percentages  provide  a  rough  indication  of  magnitude. 


accounted  for  by  the  squared  sum  and  squared  difference  of  the  location 
effects  B  and  C  which  appear  respectively  as  the  last  terms  in  the 
denominator  and  numerator  of  equation  (1).  A  natural  way  to  proceed  is  to 
compute  variances  from  the  residuals  obtained  aftor  eliminating  large  location 
effects.  After  such  elimination  the  alias  relations  of  equation  (1)  remain 
the  same  except  thet  location  affects  from  eliminated  variables  drop  out. 

That  is  seros  are  substituted  for  eliminated  variables*  Variance  analysis  for 
the  residuals  after  eliminating  effacta  of  B  and  C  are  shown  in  Figure 
2(b)*  The  dispersion  effsot  associated  with  C  (factor  IS)  is  now  correctly 
indicated  as  extrema.  It  is  shown  in  the  paper  referenced  above  how*  more 
generally#  under  circumstances  of  effect  sparsity  a  location-dispersion  model 
nay  be  correctly  identified  when  a  few  effects  of  both  kinds  are  present. 


6*  ANALYSIS  OF  UNRSPLICATEP  FRACTIONAL  DESIGNS 

Another  important  problem  in  the  enalysia  of  unreplieated  fractional 
designs  and  other  orthogonal  arraya  eonoerna  tha  picking  out  of  "aotive" 
factors.  A  serious  difficulty  is  that  with  unreplleatad  fractional  designs  no 
simple  estimate  of  the  experimental  error  varlanoa  against  which  to  judge  the 
effects  Is  evailebla. 

In  one  valuable  procedure  due  to  Cuthbert  Daniel  (195$#  1976)  affects  are 
plotted  on  Normal  probability  caper.  For  illustration  Table  4  shows  the 
calculated  effeota  from  a  2®y  design  used  in  an  experiment  on  injection 
molding  (Box#  Hunter  and  Huntar#  1978#  p.  399).  These  effeota  are  plotted  on 
normal  probability  paper  in  Figure  3* 


T1  -  -0.7  ♦  1 
Ta  -  -0.1  ♦  2 
Tj  -  5.5  ♦  3 
T4  ■  -0.3  +  4 
Ts  -  -3.8  ♦  5 
Tg  -  -0.1  ♦  6 
T7  «  0.6  ♦  7 
Tt  -  1.2  *  8 


mold  temp, 
moisture  content 
holding  pressure 
cavity  thiekness 
booster  pressure 
cycle  time 
gate  sice 
aerew  speed 


Tg  ■  *1,2  “  -0.6  * 
Tig  ■  Ti*3  -  0.9  + 
T11  "  *1,4  “  + 
*12  "  *1.5  "  4,8  * 
*13  “  *1.6  "  "°*3  * 
*14  "  *1.7  "  * 
*15  "  *1.5  "  "0e6  + 


1.2  +  3.7  +  4.8  +  5.6 

1.3  *  2.7  -I-  4.6  +  5.8 

1.4  +  2.8  +  3.6  +  5.7 

1.5  +  2.6  +  3.8  +  4.7 

1.6  +  2.5  +  3.4  +  7.8 

1.7  ♦  2.3  +  6.8  +  4.5 

1.8  +  2.4  +  3.5  +  6.7 


TABUC  4.  Calculated  effects  from  a  design  showing 

alias  structure  assuming  three  factor  and  higher  order 
interactions  negligible.  Znjeotion  molding  experiment* 


An  alternative  Bayesian  approach  (Box  and  Meyer ,  1984b)  is  as  follows i 
“  be  standardised  effects  with 


Let 


Tj_  ■  e^  if  effect  inert 

i 

T±  ■  e^  ♦  if  offset  active 


e1  ♦  N(0,o2),  tA  +  N(0,o2)  k2  - 


a  a 
o  +  c* 


Suppose  the  probability  that  an  effect  is  active  is  a. 


factors 


Let  S(r)  be  the  event  that  a  particular  set  of  r  of  the  v 
ere  active ,  and  let  .  ba  the  vector  of  estimated  affects  corresponding  to 
eotive  factors  of  a(rj.'  Then,  (Box  and  Tiao,  1968)  with  p(o)  -  £  the 
posterior  probability  that  J.  .  are  the  only  aotiva  effects  ist  9 

'  1  m  v 

■  trs^rl  V  t1*-)  * . 

k 

wh*te  B(r )  -  S( r )X( r )  And  S  -  J'J.  In  particular  the  marginal  probability 
that  an  effect  i  is  active  give  T,  a  and  k  is  proportional  to 


-1  r  ■  “  I 

l  ql  { 1  “  (1  - 


(r) 

1  active 

A  study  of  the  fractional  factorials  appearing  in  Davies  (1954);  Daniel 
(1976)  and  Box,  Hunter  and  Hunter  (1978)  suggested  that  a  might  range  from 
0*15-0.45  while  k  might  range  from  5  to  15.  The  posterior  probabilities 
computed  with  the  (roughly  average)  values,  a  ■  0.30  end  k  ■  10  are  shown 
in  Figure  4(a)  in  which  N  denotes  the  probability  (negligible  for  this 
example)  that  there  are  no  aative  effects.  The  results  from  a  sensitivity 
analysis  in  which  a  and  k  ware  altered  to  vary  over  the  ranges  mentioned 
above  is  shown  in  Figure  4(b). 

Zt  will  be  seen  that  Figure  4(a)  points  to  the  conclusion  that  active 
effects  are  associated  with  columns  3,  5  and  13  of  the  design  and  that  column 
8  might  possibly  also  ba  associated  with  an  aotiva  factor.  Figure  4(b) 
suggests  that  this  conclusion  is  very  little  affected  by  widely  different 
oholces  for  a  and  k.  Further  research  with  different  choices  of  prior, 
with  marginisation  with  respect  to  k,  and  with  different  aholces  of  the 
distribution  assumptions  is  being  conducted. 


For  three-level  and  mixed  two  and  three  level  designs  for  axample,  this 
analysis  is  carried  out  after  the  effects  are  scaled  so  that  they  all  have 
equal  variances. 
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Figure  4 (a)  Welding  experiment.  Posterior  probability  that  feotor 
1  la  active  (a  ■  0.30*  k  «  10) . 


(3)  (9)  (12) 


« 

F 

__Ll 

_il 

« 

jllI 

■I  ■■  i  rr  i — w,  i  -i.  j.  j. 

*  4  35  *  9  4  7  •  •  10  ll  12  13  14  15 


figure  4(b)  Sensitivity  analysis  for  posterior  probability 
a  •  .15  -  ,45,  k  -  5  -  15. 


7.  OTHER  RESEARCH 


Topics  which  are  emphasized  in  Taguchi's  approach  to  "off  line  quality 
control"  are  (a)  reduction  ot  variation  by  error  transmission  studies  and  (b) 
the  choosing  of  a  product  design  so  that  it  is  robust  with  respect  to 
environmental  variables* 

These  topics, are  also  receiving  attention  in  further  research* 
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ABSTRACT.  Section  794  of  the  1984  Defense  Appropriations  Act  requires 
written  guarantees  from  prime  contractors  that  weapon  systems  were  designed 
and  manufactured  to  conform  to  the  government's  performance  requirements. 
While  warranties  covering  defects  in  material  and  workmanship, and  even  product 
performance  agreements  with  contingent  liabilities  extending  up  to  5  years 
are  not  new  on  defense  equipment,  the  scope  of  this  law  far  exceeds  past 
contracting  regulations.  Since  compliance  must  bo  determined  using 
operational  experience  rather  than  controlled  test  data,  the  experimental 
design,  data  collection,  screening  and  analysis  techniques  ars  all  critical 
aspects  in  negotiating  agreements.  Technical  problems  also  arise  on  certain 
types  of  equipment  where  the  data  collected  must  be  from  a  surrogate 
population  or  eome  subset  of  the  population  whose  performance  may  be  affected 
by  non-uniform  environmental  or  operational  conditions.  ,\ 

■  it  . 

I.  INTRODUCTION  AND  CLINICAL  FOCUS.  The  aspect  that  was  moat  disturbing 
to  the  Department  of  Defense  in  Section  794  was  not  the  requirement  to,  cover 
defects  in  material  and  workmanship  but  the  ambiguous  aspect  of  the  law 
requiring  contractors  to  provide  s  written  guarantee  on  weapon  ?yst«m 
performance.  Much  of  the  initial  activity  responding  to  this  requirement 
has  centered  around  the  potential  legal  ramifications  of  contractors  accepting 
long-term  contingent  liabilities  for  tha  operational  performance  u f  their 
products.  The  underlying  technical  issue  that  lias  at  tha  heart  of  any 
commitment  to  guarantee  weapon  ayatem  performance  haa  been  largely  Ignores 
to  data.  The  basic  issue  that  needs  to  be  understood  la  hov  Joes  the 
tequlrement  dictated  by  legislation  change  the  way  in  which  the  Oovernrvnt 
determines  that  a  contractor's  product  meets  specified  performance 
requirements  in  satisfying  a  military  need.  Therefore,  the  focus  should 
really  be  on  developing  sound  experimental  designs  when  planning  contract 
strategies  so  that  an  adequate  evaluation  can  be  made  of  both  proposed 
warranty  price  as  well  as  contingent  risk  and  liabilities. 


II.  BA8I8  OF  THE  REQUIREMENT.  The  Department  of  Defense  iseued  guidance 
on  14  March  1984  to  attempt  to  clarify  the  nature  of  the  requirement.  The 
guidance  specified  that  there  would  be  two  types  of  weapon  system  guarantees, 
both  contained  within  a  single  contract.  The  first  of  thest  written 
guarantees  is  for  conformance  to  specified  performance  requirements.  Failure 
to  meet  the  guarantee  as  evident  through  either  test  or  demonstration  or 
in  operational  ute  over  some  specified  period  of  time,  would  require  the 
contractor  to  deeign  and  manufacture  the  system  to  satisfy  performance 
shortfalls  or  repair  or  replace  parts  at  no  Increase  in  price  to  the 
Government.  The  second  written  guarantee  covers  freedom  from  all  defects 


in  material  and  workmanship  applicable  Co  all  end  items  for  a  specified 
period  of  time.  Failure  to  meet  this  guarantee  would  require  repair  or 
replacement  of  components  at  no  cost  to  the  Government.  The  duration  of 
each  guarantee  may  be  different  within  a  given  contract.  There  is  a 
requirement  for  guarantee  costs  to  be  evaluated  considering  both  the  potential 
benefits  and  total  cost  to  the  Government.  Contracting  officers  may  include 
clauses  to  limit  the  contractor's  total  liability  for  guarantees  and  require 
guarantees  at  the  component  level  where  it  is  neither  cost  effective  or 
feasible  at  the  weapon  system  level. 


III.  CURRENT  ACCEPTANCE  PROCEDURES.  In  order  to  understand  the 

potential  impact  of  this  requirement  it  is  necessary  to  briefly  review  how 
the  Government  determines  acceptability  of  weapon  system  performance 
requirements  currently.  In  most  cases  the  Government  establishes  performance 
requirements  through  incorporation  into  a  system-level  specification.  Upon 

award  of  a  development  contract,  the  system  design  evolves  to  achieve  the 
performance  requirements  stated  by  the  Government.  The  systems  engineering 
process  is  the  overall  mechanism  that  translates  Government  specified 
requirements  into  a  delivered  product  useable  by  the  Armed  Forces  of  the 

United  States.  During  the  development  program  there  are  a  series  of 
configuration  audits  in  which  the  contractor's  design  approach  is  reviewed 

by  the  Government.  The  Government  implicitly  accepts  the  contractor's  design 
approach  through  an  evolution  on  ongoing  efforts  during  development  such 
as  conduct  of  paper  studies,  prototype  evaluations,  and  developmental  test 

vesulto  provided  at  scheduled  design  reviews.  At  the  functional  configuration 
audit,  the  contractor  provides  "design-to"  documentation  including  allocated 
specifications,  drawings,  test  results,  and  proposed  acceptance  test 
procedures  for  approval  by  the  Government.  Government  approval  of  the  results 
of  the  functional  configuration  audit  is  an  Indication  that  the  contractor's 
design  approach  will  meet  the  requirements  of  the  contract  Including  the 
performance  specifications.  It  should  be  noted  that  thera  is  an  implicit 
acceptance  of  the  technical  risk  associated  with  meeting  the  performance 
requirements  at  this  point  in  the  development  process.  In  similar  manner 
the  Government  determines  that  the  contractor  is  ready  for  production  through 
what  is  termed  the  physical  configuration  audit.  Information  supporting 
manufacturing  readiness  includes  manufacturing  plans,  product  assurance 
plans,  initial  operational  test  results  and  product  readiness  reviews.  Upon 
acceptance  of  the  product  baseline,  again  documentation  consisting  of 
specifications ,  teat  results,  and  acceptance  test  procedures,  the  Government 
determines  that  a  system  or  component  will  meet  the  requirements  of  the 
contract  for  production.  As  Items  are  produced,  the  Government  determines 
compliance  with  the  approved  "de»ign*to"  and  product  baseline  through 
acceptance  testing  and  Inspection.  In  summary,  except  for  latent  defects, 
fraud  or  misroprooentat ion ,  as  well  as  short-term  liability  for  defects 
in  materials  and  workmanship  covered  under  a  correction  of  deficiencies 
the  Government  becomes  the  self-insurer  with  respect  to  the  use  of  the  product 
In  the  operational  environment.  That  is,  the  Government  accepts  the  risk 
of  the  adequacy  of  design  to  meet  specified  performance  requirements. 

IV.  _ WHAT  *  S  PI FFEREMT?  A  written  guarantee  or  a  warranty  survives 

acceptance  of  the  product  by  the  Government.  That  is,  the  acceptability 
of  the  product  to  meet  specified  performance  requirements  and  be  free  from 
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defects  in  material  and  workmanship  may  extend  for  a  period  of  time  during 
the  operational  use  of  the  delivered  system.  Instead  of  determining  the 
acceptability  of  the  product  based  on  developmental  test  results  In  a 
relatively  well  controlled  environment,  the  full  acceptance  that  performance 
as  evident  through  design  and  manufacture  is  now  delayed  for  verification 
in  the  operational  environment.  The  basic  design  and  manufacturing  process 
is  the  same  but  there  could  be  some  fundamental  changes  that  may  affect 
the  development  effort  including  (l)  Government  performance  goals  may  be 
lower  to  recognize  increased  risk  in  determining  compliance  in  the  operational 
environment j  (2)  the  contractor's  design  approach  may  be  more  conservative 
using  proven  technologies  to  reduce  the  contingent  liability)  (3)  additional 
development  and  operational  testing  may  be  proposed  during  the  full-scale 
development  effort  to  reduce  the  technical  risk  of  failed  products  that 
do  not  meet  the  performance  requirements;  and  (4)  development  contract  costs 
may  increase  because  of  the  Increased  testing  efforts  noted  and  Increased 
emphasis  on  product  assurance  activities  (minimising  the  consequence  of 
significant  performance  shortfalls  of  delivered  systems  in  the  operational 
environment ) . 

An  end  result,  however,  the  contractor's  post  acceptance  liability 
increases  because  the  technical  risk  that  a  product  meets  the  performance 
requirements  of  the  contract  has  been  shifted  to  some  extent  from  the 
development  phase  into  the  post  delivery  period  for  both  specified  performance 
requirements  and  defects  In  material  and  workmanship.  A  comparison  of  key 
characteristics  that  drive  this  technical  risk  include  (1)  an  evaluation 
In  the  operational  vb  controlled  environment)  (2)  operation  by  "green  suiter" 
and  maintainers  compared  to  highly  skilled  engineers  and  technicians)  (3) 
extended  commitments  of  up  to  five  years  vs  as  low  as  90  days)  (4)  the  need 
to  use  operational  weapon  system  related  diagnostic  equipment  vs  test 
instrumentation)  and  (5)  dependence  on  standard  service  data  reporting 
information  to  determine  compliance  in  comparison  with  special  data  collection 
procedures  used  in  a  controlled  test  environment. 

V.  CLINICAL  CHALLENGE ■  As  described  above,  procedures  used  to  determine 
the  acceptability  of  products  in  the  development  process  using  the  results 
of  relatively  controlled  testing  and  Inspection  is  well  defined.  The 
requirements  generated  by  Section  794  of  the  1984  Defense  Appropriations 
Act  shift  the  determination  of  compliance  from  the  development  process  into 
the  operational  environment.  The  problem  is  to  determine  experimental  design 
guidelines  for  evaluation  of  products  in  the  operational  environment.  Some 
issues  relative  to  the  development  of  large-scale  experimental  designs  and 
anticipated  results  should  consider!  (1)  using  the  total  delivered  population 
to  collect  reeults  or  a  selected  sample  of  items  that  could  be  more  closely 
controlled)  (2)  the  desirability,  of  sequential  testing  for  risk  reduction 
throughout  the  extended  production  delivery  period  of  a  program)  (3)  methods 
used  to  relate  the  reported  performance  of  "pseudo"  products  to  the  warranted 
populatioh.  This  occurs  mainly  in  areas  where  training  devices  are  used 
during  peacetime  operations  and  war  reserve  material  may  reside  in  deep 
storagei  and,  (5)  experimental  designs  considering  the  economy  in  determining 
compliance  with  full  consideration  of  the  risk  of  accepting  poor  performance 
and  rejecting  poor  quality  products. 


SOHMA&T.  On  3  January  1985,  the  Department  of  Defense  issued  Defense 
Acquisition  Circular  84*9  as  a  temporary  regulation  on  the  requirements 
for  warranties  on  equipment  systems  in  compliance  with  the  1985  DOD 
Authorization  Act.  Extensive  warranties  are  now  a  mandatory  requirement 
for  DOD  programs.  This  continuing  requirement  creates  a  challenge  and 
opportunity  for  mathematicians,  statisticians  and  operations  researchers 
in  support  of  the  material  acquisition  process  to  provide  experimental  designs 
using  data  from  the  operational  environment  to  determine  compliance  with 
weapon  system  performance  requirements. 


A  Prioritisation  Methodology  for  Materiel  Programs 


LTC  F.  Thomas  Balser 
and 

Richard  T.  Maruyama 


Methodology  Branch 
Studies  and  Analysis  Directorate 
Deputy  Chief  of  8taff  for  Combat  Developments 
U8  Army  Training  and  Doctrine  Command 
Fort  Monroe •  Virginia 


Abstract 


The  United  States  Army  Training  and  Doctrine  Command  (TRADOC)  performs 
combat  developments  activities  for  the  Army.  The  baeia  for  establishing 
future  Army  requirements  ie  the  Mission  Area  Analysis  (MAA)  process.  MAA 
evaluate  the  capabilities  of  the  programmed  force  against  a  threat  projected 
about  five  years  beyond  the  end  of  the  current  program.  The  deficiencies 
uncovered  are  integrated  and  prioritised  in  the  Battlefield  Development  Plan 
(BDP)  to  put  them  in  the  perspective  of  the  total  battlefield.  HQ  TRADOC  has 
nov  developed  a  methodology  to  link  materiel  programs  in  the  Army's  Long  Range 
BDA  Plan  (LRRDAP)  to  BDP  deficiencies  and  prioritise  these  programs  based  on 
their  contribution  to  resolving  BDP  deficiencies.  The  methodology  employs  set 
theory  to  estimate  the  total  relative  vorth  of  each  program  and  establish  its 
relative  priority.  The  methodology  contributes  to  year-to-year  consistency  in 
priorities  that  should  yield  greater  efficiency  in  the  Army's  allocation  of 
resources  to  materiel  projects. 


The  Army's  Training  and  Doctrine  Const nd  (T1AD0C)  in  partnership  with  tha 
Amy's  Matariel  Command  (AMC)  and  BQDA  ara  forging  a  coordinated  effort  to 
improve  tha  Army's  materiel  development  and  acquisition  cycle.  TBADOC 's 
Mieeion  Area  Analysis  (MAO ,  Battlefield  Development  Flan  (BDP)  and  the  Amy's 
Long  Bangs  Kesearch  Development  and  Acquisition  Flan  (LBBDAP)  provides  the 
roadmap  for  structuring  and  equipping  the  Army  of  the  future.  These  processes 
provide  tha  means  by  which  the  Army  can  consider  future  requirements  into  the 
Planning,  Programming,  Budgeting  and  Ire cut ion  8ystea  for  resource  allocation. 

In  response  to  0MB  Circular  A-109,  AB  1000-1  tasks  TBADOC  with  conducting 
Mission  Area  Analysis  to  analytically  support  requirements  for  future  Amy 
doctrine,  training,  organisational  structure,  and  materiel*  As  part  of  the 
Concept  Beeed  Acquirements  System  (CBBO) ,  TBADOC  has  divided  the  wartime 
Battlefield  mission  into  13  mission  arqas,  These  mission  areas  serve  as 
building  blocks  for  measuring  the  capabilities  of  the  progremed  force  in  the 
current  Program  Objectives  Memorandum  (POM)  to  fight  a  auccaaaful  battle 
against  a  threat  projected  about  10  years  into  the  future.  These  MAA  and 
their  resulting  deficiencies  eatsbliSh  the  battlefield  needs/requirements,  BQ 
TBADOC  in  conjunction  with  its  schools  and  centers  and  other  major  Amy 
commands  integrates  and  prioritises  these  deficiencies  across  all  mission 
areas  into  a  single  ordered  list  representing  the  broader  perspective  of  the 
total  battlefield.  In  prior  years,  the  BDP  was  an  end  product.  While 
indirectly  influencing  the  priorities  of  doctrinal  literature,  training 
programs,  force  structure  modif lections,  or  materiel  developments  there  was  no 
direct  linkage  which  would  provide  an  analytically  baaed  rationale  for  the 
relative  priorities  of  any  of  the  Amy's  future  developmental  activities.  In 
1984,  TBADOC  set  out  to  correct  this  anomaly  by  developing  a  methodology  to 
establish  the  materiel  priorities  for  battlefield  systems  in  the  Army's  Long 
Bangs  Beseerch,  Development,  and  Acquisition  Plan  (LBBDAP)  based  on  their 
contribution  toward  resolving  one  or  more  of  the  deficiencies  in  the  BDP.  The 
establishment  of  such  a  methodology  will  lend  stability  and  crediability  to 
the  priorities  of  the  associated  materiel  programs  while,  at  tha  same  time 
provide  the  audit  trail  back  to  tha  supporting  analysis  against  the  projected 
threat.  This  methodology  will  therefore  extend  the  total  concept  based 
requirement  system  into  the  PPB18  system  (see  figure  1).  The  quest  for  on 
appropriate  linkage  methodology  was  triggered  by  TBADOC  Begulation  11-9  which 
requires  that  the  Studies  and  Analysis  Directorate  (liAD),  Deputy  Chief  of 
Itaff  for  Combut  Developments  (DC8CD)  develop  such  a  methodology  for 
prioritising  the  LBBDAP  bised  on  BDP  deficiencies.  This  paper  highlights  the 
thcoritical  underpinnings  and  general  procedures  which  resulted  from  this 
methodology  development* 
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II.  Methodology . 

The  priority  of  materiel  program  should  bo  diractly  ralatod  to  tha 
priority  of  tha  BDP  dafieianoiaa  thay  eorract.  Matarial  programs  that  oorraot 
an  important  battlafiald  daficiancy  should  bs  highs r  on  tha  matarial  priority 
list.  This  mathodology  attampta  to  diractly  aatabliab  this  link*  Baeausa  of 
tha  broad  and  ganaral  natura  of  existing  BDP  dafieianoiaa  it  ia  likaly  that 
multipla  matarial  programs  ara  associatad  with  aaeh  daficiancy.  On  tha  other 
hand,  tha  aama  matarial  program  could  contribute  to  tha  solution  of  more  than 
ona  daficiancy.  This  complication  is  considered  aud  this  methodology  will 
make  tha  necessary  adjustments  for  those  cases.  An  example  is  show  in 
figure  2, 


BDP  Def  # 

123456  ...  76 


Matarial 

Programs 
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In  thi«  case,  program  A  contribute*  to  the  reeolution  of  BDP  deficiencies 
number  3  and  76  while  program  C  contribute*  to  four  deficienciee  and  program  E 
to  only  one,  Aa  va  will  eetabliah  later  in  the  paper  thia  doea  not  in  itaelf 
juatify  that  program  C  haa  mora  banefit  to  the  Any,  In  the  example,  program 
I  ia  the  only  program  which  contributea  to  the  resolution  of  deficiency  4 
while  each  of  the  deficiencies  affected  by  program  C  also  has  other  programs 
which  make  a  contribution.  Aa  an  additional  factor  in  understanding  the 
mathematical  foundation  of  the  methodology,  one  must  ali?o  consider  the  nature 
of. the  BDP  deficiencies  which  drive  the  prioritisation  effort, 

BDP  deficiencies  are  aggregations  of  specific  MA  deficiencies  and  aa  such 
are  seta  of  commonly  related  battlefield  problema.  While  there  are 
approximately  1400  apecific  MA  deficienciea  in  the  MAA  proeeaa,  theae  are 
aggregated  into  227  BDP  deficienciea  which  are  them  prioritised  by  a  large 
group  of  general  officers.  Thia  group  ia  repreaantative  of  all  combat 
missions  of  the  Army.  Figure  3  shows  the  relationship  of  the  1400  apecific  MA 
deficiencies  which  are  aggregated  into  the  227  BDP  deficiencies  for 
prioritisation  by  the  general  officers. 


MISSION  AREA  DEFICIENCIES  1400 


Became  BDP  deficiencies  are  aggregations  of  specific  mission  area 
deficiencies  end  each  deficiency  could  require  nultiple  progress  for  its 
resolution,  the  mathematical  concept  of  sets  is  appropriate.  Baring 
established  a  set  of  materiel  programs  for  the  resolution  of  each  BDP 
deficiency  (which  itself  ie  a  aet  of  the  sore  specific  XU  deficiencies)  the 
methodology  uses  an  underlying  assumption  that  to  the  extent  a  materiel 
program  contributes  to  multiple  BDP  deficiencies,  these  deficiencies  here 
overlapping  characteristics  and  there  ia  some  degree  of  commonality  among 
them. 

The  example  (figure  4)  illustrates  this  idea.  That  is,  if  a  materiel 
program  contributes  to  the  solution  of  three  BDP  deficiencies  (BDP#1,  BDP#3 
and  BDPdll) ,  then  by  fielding  this  program  we  are  simultaneously  solving  a 
fraction  of  all  three  deficiencies.  The  methodology  must  not  over  credit 
a  program's  battlafield  contribution  by  assuming  that  the  aet  is  disjoint. 


PROGRAM  CONTRIBUTION 


PROBLEM: 

WHAT  It  THE  VALUE  OF  A  MATERIEL  PROGRAM 
THAT  CONTRIBUTES  DIFFERENT  AMOUNTS  TO 
DIFFERENT  DEFICIENCIES? 


E.6.  PROGRAM  X  CONTRIBUTES: 
1.6  TO  BDP  p  1 
0.4  TO  BDP  *  3 
0.1  TO  BDP  #11 


figure  4 

The  question  then  arises  about  the  degree  of  overlap.  Since  there  is  no 
feasible  method  of  quantitatively  measuring  this  overlap,  existing  procedures 
assume  that  it  ia  propotional  to  the  degree  of  contribution  of  the  program  to 
each  deficiency.  This  assumption  appears  to  be  not  only  reasonable  hut  may  be 
also  quite  accurate.  If  a  program  makes  a  very  large  contribution  to  two 
different  deficiencies,  they  should  show  a  significant  degree  of  overlap  in 
the  deficient  battlefield  functions  addressed.  On  the  other  hand,  a  program 
with  a  very  minor  contribution  to  two  (or  auny)  deficiencies  would  establish 
only  a  small  degree  of  overlap. among  the  deficiencies  in  question.  The 
mathematics  to  handle  these  phenomena  utilise  the  well  know  uniotr operations 
applied  to  sets.  In  the  example  of  figure  4  program  Z  contributes  0.6,  0.4, 
and  0.1  to  the  three  deficiencies.  Therefore,  tbs  combined  eolution  _ 
(correction)  to  the  three  deficiencies  is  0-.78  not  1 .10  (.6  +.4  ♦.!  ■>*  1.1), 
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combined  contribution-  ( .6)+(  .4)+(  .l)-(  ,6)(  .4)-(  .6)  C  .l)-(  .4)  ( .1) 

♦<.6>(.4)(.1> 

-  0.784 

Another  desired  feature  of  the  methodology  mould  account  for  the  fact  that 
aooe  materiel  program*  contribute  to  higher  priority  deficienciee  (or  groupa 
of  deficienciee)  than  other* •  For  example,  if  two  materiel  program* 
contribute  an  equal  amount  to  different  *et»  of  deficiencies,  the  one 
contributing  to  the  higher  priority  eat  of  deficienciee  should  have  more 
battlefield  worth.  This  is  illustrated  for  programs  Z  and  I  in  figure  5. 

Both  programs  have  the  same  amount  of  contribution  (.6  and  .1)  but  the  eat  of 
deficiencies  affected  by  program  Z  is  prioritised  higher  than  the  set  affected 
by  program  T.  In  this  case  program  Z  mould  be  prioritised  higher. 


Program  Z 

BDP  A  Contribution 

16  .6 

36  .1 


Program  1 

BDP  * _ Contribution 

62  .6 

84  .1 


figure  5 


In  order  to  resolve  this  complication  a  deficiency  weight  must  be  assigned  to 
the  BDP  deficiency  that  reflects  its  relative  priority  position.  For  the  BDP 
84  prioritisation  a  uniform  distribution  of  weights  mas  used  ranging  from  0  to 
1.  This  mathematical  assignment  was  made  according  to  the  following  formula. 

Qi  ■  1  -  jLtJQjJL  (1> 

n 

In  the  case  where  n  ■  227  and  where  i  ■  1.  2.  3 .  227.  Qj,  Q3,  end  Qu 

take  on  weighted  valuea  of  0.997.  0.988.  and  0.953.  Those  deficiency  weights 
when  applied  to  the  contribution  estimate  results  in  a  priority-adjusted 
contribution  value  of  0.780  (figure  6)  for  program  Z.  Therefore,  materiel 
programs  that  effect  BDP  deficiencies  of  high  priority  will  be  parf erred  over 
materiel  programs  that  effect  lower  priority  deficiencies. 

Assigning  a  numerical  value  to  the  degree  of  contribution  of  a  program  for 
a  deficiency  is  another  subjective  area  open  to  debate  and  future  research. 
After  some  preliminary  research  and  trial  applications  into  different  scales, 
the  authors  arrived  at  a  five  point  scale  using  the  narrative  descriptors 
uhown  in  table  1  and  values  of  0.8.  0.6,  0.4,  0.2.  and  0.1  for  categories  A 
through  B  respectively. 

These  five  categories  will  discriminate  enough  to  ensure  that  materiel 
programs  making  a  significant  contribution  to  high  priority  deficiencies  will 
receive  favorable  treatment  in  the  competition  for  funding* 
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SET  THEORY 


XiUXilIXt  *•  Xi-t-  Xi  +  Xt —  (Xm  Xi) 

-  (Xm  Xc)  -  (Xm  Xt) 

4-  (XmXln  Xt) 

t»,ny  [.G.  V(P)x  -  (.997X.6)  +  (.933X.4)  +  (.9S3X.1) 

-  (.S97X.6X-9S8X.4)  -  (-997X.9X.9S3X.1) 

-  (.998X.4X.953X.1) 

+  (-997X6X-988X.4X.9S3X.1) 

-.790 


figur*  6 


Tab It  I 


CONTRIBUTION  CATEGORIES 

A.  ABSOLUTELY  ESSENTIAL  TO  SOLUTION  OF  DEFICIENCY 

B.  MAJOR  CONTRIBUTOR  TO  SOLUTION  OF  DEFICIENCY 

C.  SUBSTANTIAL  CONTRIBUTION  TO  SOLUTION  OF  DEFICIENCY 

D.  SMALL,  YET  STILL  DIRECT  CONTRIBUTION  TO  SOLUTION  OF 
DEFICIENCY 

E.  LIMITED  OR  INDIRECT  CONTRIBUTION  TO  SOLUTION  OF 
DEFICIENCY 


rtV.AVtY.n 


The  Bithenetica  of  set  theory  deacri'ani  above t  the  BDP  priority  weight  and 
the  contribution  scale,  establish  the  materiel  program  relative  worth*  This 
formulation  ean  be  generalised  in  the  following  for os 

GENERALIZED  FORMULA 


V(P)I=  Z  {-l)r  l  i  ir  Aji 

WHERE: 

V|P)l  -  TOTAL  RELATIVE  WORTH  OF  PROGRAM  I 

n  -  NUMBER  OF  PROGRAMS  CONTRIBUTING  TO 
DEFICIENCY  j 

Aji  -  PRODUCT  OF  THE  POSITION  VALUE  FOR 
DEFICIENCY  I  TIMES  THE  CONTRIBUTION 
OF  PROGRAM  I  TO  DEFICIENCY  j 
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The  following  example  la  an  application  of  the  methodology  applied  to  three 
notional  materiel  programs • 


EXAMPLES 


PROGRAM  X 

IDP  CONT 
PIN  VALUE  VALUE 

.1  .191  J 


PROGRAM  Y. 

IDP  CONT 
PSN  VALUE  VALUE 

«  .177  .1 

14  .143  .4 

37  .145  .1 


PROGRAM  Z 

IDP  CONT 


16  .721  .1 

142  .319  .4 

292  .143  .1 


Vp  -  A  4*  B 
Vp  -  .798  - 


C  -  (AB)  -  (AC) 
Vp  -  .875 


(BC)  +  (ABC) 

Vp  *=  .306 


figure  7 


This  example  demonstrates  that  program  T  ia  prafarad  ovar  program  Z  and 
program  Z  because  program  T  contributaa  to  three  daficianciaa  in  tha  top  20 
paroant  of  tha  ovar all  Hat.  It  ahould  ba  pointad  out  at  thia  point  that  tha 
numbara  ganaratad  by  thia  mathodoiogy  to  prioritiaa  programa  rapraaant  ordinal 
rathar  than  cardinal  ranfcinga.  Because  tha  daficiancy  valuaa  ara  aealad 
uniformly  from  0  to  1  and  tha  contribution  vaighta  ara  subjectively 
datarminadi  tha  raaulting  calculation  produce  numbara  without  acalar 
magnitude.  That  ia  a  program  with  a  value  of  0.8  ia  more  important  than  one 
with  a  value  of  0.4  one  cannot  conclude  that  it  ia  twice  aa  important.  In  tha 
prioritiaation  of  material  programa  it  ia  auffieiant  to  determine  only  tha 
relative  order  of  tha  candidate*}  it  ia  not  nacaaaary  to  maaaura  tha  interval 
between  them.  Kaaearch  ia  currently  underway  to  invaatigata  any  benafita  that 
nay  accrue  by  trying  to  produce  a  liat  with  more  acalar  properitiaa.  Tha 
atapa  in  tha  prioritisation  mathodoiogy  need  in  1984  to  prioritise  tha  LRRDAf 
ara  summarised  in  Table  IZ.  While  tha  first  three  atapa  have  bean  described 
in  soma  detail  above,  tha  last  thraa  need  a  few  notes  of  clarification. 

Having  established  an  initial  prioritisation  of  programs  using  tha  stated 
approach,  tha  mathodoiogy  attempts  to  account  for  the  differences  in  time 
(near  term,  mid  term,  far  term),  R&D  and  procurement  costs  (high,  medium, low) 
and  developmental  risk  (high,  medium,  low).  While  mathematics  have  been 
developed  in  the  program  to  consider  these  factors,  time  and  the  availability 
of  input  data  did  not  permit  their  use  in  last  years  implementation  of  the 
priorities  program,  further  reaearch  ia  underway  to  evaluate  different 
methods  of  considering  these  factors  and  also  the  manpower  impacts  and  tha 
amount  of  investment  already  sunk  into  each  program. 

frocurement  program  prioritisation  had  to  also  consider  what  is  referred 
to  aa  "base  case"  programs  because  the  mission  area  analysis  studies  took  as  a 
given  or  base  ease  all  those  programs  currently  contained  in  the  Program 
Objectives  Memorandum  (the  Arity's  5  year  planning  and  programming  document), 
new  deficiencies  for  tha  BDP  ware  not  generated  to  support  continued 


TABLE  II 

1.  DISTRIBUTE  DEFICIENCIES  UHIFOUILY  BETW1EH  0  and  1 

2.  DETERMINE  COH8TRIBUTION  OF  PROGRAMS  TO  EACH  DEFICIENCY 

3.  DETERMINE  CUMULATIVE  CONTRIBUTION  FOR  EACH  PROGRAM 
(CONSIDER  INTERACTIVE  EFFECTS) 

4.  ADJUST  CCNTRXBUTIOI  BY  TIME,  COST,  RISK,  MANPOWER,  AND 
SUNK  INVESTMENT 

3.  PRIORITIZE  PROGRAMS 

6.  PIN  TO  "BASE  CASE"  LIST  (PROCUREMENT  PROGRAMS  ORLY) 


procurement  of  these  programs.  Since  they  vould  have  no  calculated  program 
value,  they  maintained  their  original  LREDAP  priority  and  the  prioritiaed  list 
of  newer  progress  was  integrated  or  "pinned"  into  this  list,  Ion-battlefield 
progress  outside  of  TRADOC  responsibility  were  also  handled  like  base  case 
progress. 

Although  developed  in  only  a  abort  period  of  tine  to  be  used  In  last 
year1 a  LRBDAP  prioritisation  process,  this  methodology  demonstrated  the 
feasibility  of  linking  materiel  progrea  priorities  to  the  deficiencies 
contained  in  the  battlefield  development  plan.  Manual  or  "cosnon  sense" 
adjustsents  had  to  be  sade  for  example  to  align  the  6,3B  and  6,4  components  of 
the  ease  program  to  place  then  in  the  seme  funding  band  (funded,  at  risk, 
unfunded).  In  the  end,  of  course,  the  list  was  subjected  to  expert  review  at 
the  0-6,  0-8,  0-9.  and  0-10  levels ,  Of  the  over  300  RDT1  programs  prioritised 
with  this  hasty  sethodology,  more  than  65Z  retained  their  relative  poaition 
and  rank  in.  the  final  approved  DA  Long  Range  Plan  (figure  8), 

OVER  300  RDT&E  PROGRAMS  WHERE  PRIORITIZED 
THIS  YEAR.... 


figure  8 


Research  continues  in  the  areas  described  throughout  this  paper  and  on 
various  means  to  maintain  a  relative  consistency  of  program  priorities  from 
year  to  year.  Contributors  desiring  additional  information  or  wishing  to 
consent  on  proposed  improvements  to  this  process  are  encouraged  to  contract 
the  authora  by  phoning  (804)  727-3004  or  by  writing  HQ  TRADOC,  ATTMt  ATCD-AM, 
Pt  Monroe,  VA  23631-3000. 
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ABSTRACT 

Two  statistics,  denoted  by  and  a  q-value,  are  proposed  to  provide 
flexibility  and  Inferential  Information  during  and  after  hypothesis  testing. 
These  two  statistics  supplement  the  well-established  statistics  normally 
denoted  by  beta  and  the  p-value,  The  purpose  of  Pj  Is  to  emphasize  the  true 
value  of  beta  after  the  design  of  the  hypothesis  test  but  before  or  during  the 
obtaining  of  data.  A  q-value,  In  conduction  with  the  p-value  and  associated 
power  curves,  has  potential  uses  In  (1)  overruling  the  decision  of  the 
hypothesis  test,  (2)  ordering  additional  testing,  and  (3)  procurement  cost 
analysis. 

INTRODUCTION 

During  the  development  of  statistical  Inference,  there  were  two  distinct 
needs  and  resulting  philosophies.  A  statistical  technique  was  needed  to 
estimate  the  bounds  of  capabilities  of  existing  equipment  or  processess. 
A  different  statistical  technique  was  needed  to  test  the  conformity  of  newly 
procured  equipment  and/or  new  processes  to  required  specifications.  The  first 
of  these  needs  led  naturally  to  flexible  Ideas  such  as  confidence  intervals. 
The  second  was  met  by  the  rigid  procedure  of  hypothesis  testing. 


STANDARD  HYPOTHESIS  TESTING 


The  standard  philosophy  used  In  hypothesis  testing  Is  to  (1)  assume  that 
some  parameter  of  a  proposed  piece  of  equipment  or  process  conforms  to  some 
standard  and  (2)  abandon  that  assumption  only  If  there  Is  sufficient  evidence 
that  the  assumption  Is  highly  unlikely.  The  Initial  assumption,  which  Is 
called  the  null  hypothesis  and  denoted  by  H0,  Is  expressed  In  an  equation 
relating  the  physical  parameter  being  tested  arid  the  specification  value  of 
that  parameter.  The  significance  level,  denoted  by  alpha,  Is  the  minimum 
acceptable  probability  that  the  actual  test  data  could  have  come  from  a 
population  for  which  tha  null  hypothesis  Is  true.  Physical  measurements  give 
a  standard  for  evaluating  the  reasonableness  of  H0  as  a  description  of  the 
equipment  or  process  under  test. 

Alpha  Is  one  of  two  risks  In  hypothesis  testing.  It  Is  the  risk  of 
rejecting  the  null  hypothesis  In  favor  the  alternate  hypothesis,  denoted  by 
Ha,  when  the  null  hypothesis  Is  true.  The  companion  risk  to  alpha,  which  Is 
denoted  by  beta,  Is  the  probability  of  not  rejecting  the  null  hypothesis  when 
the  null  hypothesis  Is  not  true.  Beta  depends  on  four  parameters  which  must 
be  established  jointly  by  the  statistician  and  the  manager  of  the  equipment  or 
process  being  tested: 

(1)  Alpha. 

(2)  The  specification  value  of  the  parameter  being  tested  as  stated  In 
the  null  hypothesis. 

(3)  A  specific  value  of  the  parameter  satisfying  the  alternate 
hypothesis. 

(4)  The  number  of  proposed  physical  measurements  of  the  parameter  being 
tested. 


Beta  Is  found  from  the  statistics  function  which  describes  the 
distribution  of  the  test  statistic  under  various  alternatives.  Since  there 
are  many  values  of  the  parameter  being  tested  that  do  not  meet  the 
specification,  there  are  many  betas.  The  power  curve  Is  defined  as  the  graph 
of  one  minus  beta  versus  values  of  the  parameter  being  tested.  Since  one 
minus  beta  Is  the  probability  of  rejecting  the  null  hypothesis  when  the  null 
hypothesis  Is  not  true,  the  power  curve  Is  defined  more  generally  as  the  graph 
of  the  probability  of  rejecting  versus  values  of  the  tested  parameter.  Alpha 
and  beta  have  several  commonly  used  names  Including  type  1  and  type  II  risks, 
the  producer's  and  consumer's  risks,  and  the  government's  and  contractor's 
risks.  The  second  and  third  sets  of  names  are  obviously  Interpretative  and 
explanatory  when  hypothesis  testing  Is  applied  In  procurement  actions.  Alpha 
and  beta  are  competing  risks;  as  either  of  them  Increases,  the  other 
decreases. 

In  the  operating  procedure  traditionally  used  In  hypothesis  testing,  there 
are  three  rigidly  ordered  steps; 

(1)  Formulate  a  statistical  test. 

(2)  Obtain  data  from  a  physical  test. 

(3)  Use  the  data  and  the  statistical  test  to  either  reject  or  not  reject 
the  null  hypothesis. 

The  first  step  Is  called  the  design  of  the  experiment;  this  statistical 
design  procedure  consists  of  six  sub-steps  which  are  performed  by  the 
statistician  In  consultation  with  the  manager; 

(1)  State  the  null  hypothesis. 

(2)  State  the  alternate  hypothesis. 

(3)  Select  a  value  for  alpha. 


(4)  Determine  a  rejection  rule  form  the  appropriate  statistical 
distribution  function  which  describes  the  parameter  being  tested. 

(5)  Select  a  value  for  beta  from  some  particular  non-specification  value 
of  the  parameter  being  tested. 

(6)  Determine  the  number  of  measurements  that  must  be  made  on  the 
parameter  being  tested. 

In  actual  practice,  there  are  two  common  modifications  during  the 
formulation  of  the  statistical  test.  A  range  of  non-specification  values  of 
the  parameter  being  tested  Is  normally  considered  In  the  determination  of  the 
number  of  measurements  necessary  to  achieve  the  selected  beta.  This  procedure 
requires  the  use  of  engineering  judgment  to  determine  the  range  of  values  on 
the  power  curve.  A  modification  of  both  alpha  and  beta  Is  often  necessary  to 
formulate  the  statistical  test  with  a  reasonable  and  achievable  number  of 
measurements  In  the  physical  test. 

In  classical  hypothesis  testing,  the  physical  test  Is  performed  inde¬ 
pendently  of  the  formulated  statistical  test;  no  analysis  of  physical  test 
data  is  made  until  all  measurements  are  made.  As  hypothesis  testing  Is 
actually  practiced,  the  data  Is  often  modified  before  the  final  analysis;  that 
Is,  outliers  In  the  data  are  often  discarded  by  a  process  which  may  be 
systematic  or  logical. 

In  traditional  hypothesis  testing,  the  decision  to  reject  or  not  reject 
the  null  hypothesis  Is  based  on  the  retained  data  and  the  decision  rule  from 
the  statistical  test.  Once  the  decision  Is  made  to  reject  or  not  reject,  tho 
process  Is  complete.  No  consideration  Is  given  to  the  margin  of  passing  or 
falling  and  no  questions  are  cons1d<‘-  i  concerning  possible  modifications  of 
the  statistical  test. 
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MODIFICATION  OF  ALPHA 


The  application  of  hypothesis  testing  using  the  traditional  and  rigid 
method  outlined  In  the  previous  section  has  a  major  practical  obstacle.  It 
provides  managers  with  no  guidance  other  than  to  reject  or  fall  to  reject  the 
null  hypothesis.  While  this  might  seem  sufficient  to  a  statistician  who  Is 
lead  to  believe  that  the  statistical  test  properly  considers  all  aspects  of 
the  proposed  equipment  or  process,  the  manager  may  well  find  this  "go  -  no  go" 
guidance  sorely  lacking.  The  manager  may  have  to  consider  the  effects  of  more 
than  one  parameter,  the  political  Influence  of  pressure  groups,  and  the 
economic  Impact  of  Implementing  the  new  equipment  or  process.  A  manager 
facing  these  problems  needs  all  the  help  that  the  statistician  can  provide. 
While  the  statistician  cannot  solve  the  manager's  problems,  he/she  can  at 
least  provide  more  Information  about  one  parameter  than  a  bare  "go  -  no  go" 
recommendation.  This  Is  done  by  a  modification  of  alpha  based  on  the  data 
from  the  physical  test. 

The  p-value  of  a  hypothesis  test  Is  defined  as  the  value  that  alpha  would 
have  been  necessary,  1r  the  original  statistical  test,  to  make  the  result  of 
the  decision  rule  Indecisive.  It  Is  the  value  of  alpha  which  would  act  as  a 
pivot  between  rejecting  and  not  rejecting  the  null  hypothesis;  It  Is 
calculated  using  the  data  from  the  physical  test.  It  provides  the  manager, 
who  must  consider  all  factors  and  make  the  final  decision,  with  a  measure  of 
the  degree  of  certainty  of  the  bare  "go  -  no  go"  recommendation  from  the 
hypothesis  tost.  If  a  manager  wishes  to  make  a  final  decision  which  Is  the 
opposite  of  the  result  of  using  data  from  tl.e  physical  test  In  the  decision 
rule  of  the  statistical  test,  the  p-value  provides  him/her  with  a  measure  of 
the  risk  that  must  be  taken  to  over-ule  the  judgment  of  the  hypothesis  test. 


The  existence  of  the  p-value  Is  certainly  not  new;  however,  Its  usage  has 
Increased  In  recent  years  because  managers  have  needed  more  flexibility. 
Reporting  the  p-value  to  managers,  who  make  final  decisions,  has  the  effect  of 
loosening  the  rigor  of  the  formal  hypothesis  testing.  The  p-value  provides 
the  final  decision  maker  with  more  flexibility;  it  also  tells  him/her  how 
flexible  the  result  of  the  hypothesis  test  Is.  Very  low  values  of  the  p-value 
Imply  that  the  null  hypothesis  can  be  rejected  only  If  the  producer  Is  allowed 
a  very  low  risk.  Very  high  values  of  the  p-value  Imply  that  the  null 
hypothesis  cannot  be  rejected  unless  the  producer  Is  forced  to  take  a  very 
high  risk.  Values  of  the  p-value  that  are  near  the  alpha  of  the  original 
statistical  test  Imply  that  the  null  hypothesis  can  be  rejected  If  the 
producer  takes  a  risk  near  the  original  alpha. 

PROPOSED  MODIFICATION  OF  BETA 

The  modification  of  alpha  as  outlined  In  the  previous  section  is  well 
established.  In  this  section,  modifications  to  beta  and  the  power  curve  are 
proposed  to  further  Increase  the  flexibility  and  reported  Information  of 
hypothesis  testing.  These  proposals  are  motivated  by  the  desire  to  aid 
decision  makers  as  much  as  possible. 

There  are  two  critical  parameters  which  may  logically  Initiate  a 
modification  of  beta  after  the  standard  hypothesis  test  Is  completed.  One  Is 
the  p-value;  It  will  differ  greatly  from  the  alpha  of  the  original  statistical 
test  If  the  data  from  the  physical  test  leads  to  an  extermly  strong  recom¬ 
mendation  to  either  reject  or  not  reject  the  null  hypothesis.  The  other 
critical  parameter  Is  the  number  of  measurements  actually  made  on  the  physical 
parameter  being  tested.  It  may  be  larger  than  the  number  proposed  by  the 


statistical  test,  but  the  more  common  change  Is  for  It  to  be  smaller.  In 
actual  practice,  the  number  of  measurements  In  the  physical  test  Is  often 
lowered  by  constraints  of  cost,  time,  and/or  limitations  of  personnel, 
facilities,  and/or  equipment. 

The  two  critical  parameters  result  In  two  logical  modifications  of  beta. 
These  modifications  are  denoted  by  and  a  q-value  In  this  section.  They  are 
obtained  from  the  same  basic  mathematical  algorithm  that  yielded  the  original 
beta  of  the  statistical  test,  but  different  Inputs  are  used  In  the  algorithm. 

The  statistic  denoted  by  Is  defined  as  the  result  of  the  beta 
calculation  in  the  design  of  the  statistical  test  when  the  number  of 
measurements  actually  made  1$  used  Instead  of  the  number  that  was  planned. 
The  same  alpha,  specification  value  of  the  parameter  being  tested,  and  actual 
value  of  the  parameter  being  tested  are  used  In  the  calculation  as  were 
used  In  the  original  beta  calculation;  only  the  number  of  measurements  Is 
changed.  The  numerical  value  of  0T  will  be  higher  than  the  value  of  the 
original  beta  If  the  number  of  actual  measurements  Is  lower  than  planned  by 
the  statistical  test. 

The  statistic  denoted  by  Sj  Is  the  consumer's  true  risk  when  the  data 
actually  available  from  the  physical  test  Is  analyzed.  The  "sub  T"  notation 
Is  used  to  emphasize  that  0y  Is  based  on  the  true  number  of  measurements  In 
the  actual  physical  test,  A  manager  can  use  as  a  measure  of  how  badly  a 
hypothesis  test  would  be  damaged  If  he/she  Is  pressured  Into  changing  the 
number  of  measurements  In  the  planned  physical  test.  Pressure  for  such  a 
change  can  occur  between  planning  and  testing  or  during  testing.  Naturally 
the  manager  is  concerned  about  more  than  one  possible  true  value  of  the  tested 
parameter.  Ylrb  means  that  he/she  needs  the  entire  power  curve  bused  on  3y. 


Of  course,  this  might  have  been  one  curve  In  a  family  of  curves  that  was  used 
to  design  the  statistical  test.  Unfortunately,  the  manager  might  not  have 
access  to  or  knowledge  of  this  previously  calculated  curve.  The  statistic  @T 
is  proposed  to  rectify  the  managers'  lack  of  Information,  The  decision  maker 
needs  a  power  curve  based  on  $7  at  the  time  that  pressure  exists  to  change  the 
number  of  measurements.  Statistical  advice  may  be  needed  after  the 
statistical  test  Is  formulated  but  before  the  data  from  the  completed  physical 
test  Is  analyzed.  The  statistician  should  provide  the  decision  maker  with 
relevant  values  of  between  the  formulation  of  the  statistical  test  and  the 
analysis  of  data  from  the  physical  test. 

A  q-value  Is  defined  as  the  result  of  the  beta  calculation  In  the  design 
of  the  statistical  test  when  two  changes  are  made.  A  q-value  Is  calculated 
from  the  p-value  Instead  of  the  original  alpha  and  from  the  number  of  measure¬ 
ments  actually  made  whether  or  not  this  is  the  number  that  was  planned  In  the 
original  statistical  test.  The  same  specification  value  and  actual  value  of 
the  parameter  being  tested  are  used  In  a  q-value  calculation  as  In  the 
original  beta  calculation.  Since  there  are  many  possible  values  of  the 
parameter  being  tested,  there  are  many  q-values.  Use  of  the  same  possible 
parameter  value  that  was  used  In  the  beta  calculation  allows  direct  comparison 
between  a  q-value  and  beta.  A  q-value  tends  to  be  higher  than  the  original 
beta  If  the  number  of  actual  measurements  Is  lower  that  planned  or  If  the 
p-vnlue  Is  lower  than  the  original  alpha.  Slmllarlly,  more  measurements  than 
planned  or  data  making  the  p-value  higher  than  the  original  alpha  tends  to 
make  a  q-value  lower  than  the  original  beta. 

A  q-value  Integrates  Information  about  both  the  number  of  measurements 
actually  made  In  the  physical  test  ana  the  results  of  these  measurements. 


A  q-value  has  three  potential  uses: 

(1)  Provide  Information  about  beta  If  alpha  Is  changed  to  the  p-value 
after  the  hypothesis  test  In  order  to  make  the  test  Indecisive. 

(2)  Indicate  the  need  for  additional  testing  to  make  the  test  more 
decisive, 

(3)  Consider  as  a  rating  factor  for  the  payment  in  procurement  cost 
analysis  If  the  consumer  accepts  equipment  or  services  that  fall  the 
hypothesis  test. 

If  a  decision  maker  has  Influences  which  contradict  the  result  of  a 
completed  hypothesis  test,  there  are  two  contrasting  situations  which  are  most 
Intelligently  considered  by  using  both  the  p-value  and  a  q-value.  If  the 
decision  rule  results  In  a  recommendation  to  reject  the  null  hypothesis,  alpha 
must  be  lowered  to  the  p-value  If  the  hypothesis  test  Is  to  be  viewed  as 
Indecisive  and  the  other  Influences  are  to  have  no  opposition.  In  this 
situation,  beta  for  a  particular  non-specification  value  of  the  parameter 
being  tested  must  be  raised  to  a  q-value.  In  the  opposite  situation,  the 
decision  rule  results  In  a  recommendation  to  not  reject  the  null  hypothesis. 
In  this  case,  alpha  must  be  raised  to  the  p-value  and  beta,  for  a  given  value 
of  the  tested  parameter,  must  be  lowered  to  the  q-value  In  order  to  make  the 
hypothesis  test  Indecisive  so  the  other  Influences  have  no  opposition  to 
rejecting  the  null  hypothesis.  In  both  of  these  situations,  the  decision 
raker  should  be  aware  of  the  changes  In  both  alpha  and  beta.  Thus  the  statis¬ 
tician  should  report  both  the  p-value  and  a  q-value.  Of  course,  the  decision 
maker  Is  often  Interested  In  more  than  one  possible  given  value  of  the  tested 
parameter;  thus  the  statistician  should  report  the  entire  power  curve  based  on 
q-values. 
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Increasing  the  amount  of  physical  testing  always  reduces  the  risks  In 
hypothesis  testing;  the  amount  of  additional  testing  after  a  hypothesis  test 
Is  completed  may  be  determined  from  the  power  curve  based  on  q-values  and 
other  power  curves  based  on  the  p-value  and  proposed  Increases  In  the  number 
of  measurements  of  the  parameter  being  tested.  If  a  decision  maker  Is  willing 
to  accept  the  lowering  of  alpha  to  the  p-value  but  Is  hesitant  about  raising 
beta  to  a  q-value,  more  testing  Is  necessary.  The  amount  of  additional 
testing  Is  determined  by  the  consumer's  risk  that  the  manager  Is  willing  to 
take.  Once. that  consumer's  risk  Is  determined  for  relevant  non-specification 
values  of  the  parameter  being  tested,  power  curves  may  be  used  to  determine 
the  amount  of  additional  testing.  The  additional  number  of  measurements  which 
the  manager  Is  ordering  is  the  difference  between  the  total  number  determined 
by  the  power  curves  and  the  number  of  measurements  In  the  Initial  hypothesis 
test.  The  statistician  should  report  power  curves  based  on  q-values  cal¬ 
culated  from  the  p-value  and  both  (1)  the  number  of  measurements  actually  made 
and  (2)  proposed  Increases  In  the  number  of  measurements.  This  Information 
allows  the  manager  to  evaluate  the  return  from  additional  testing. 

When  a  piece  of  equipment  or  a  service  Is  procured,  the  procuring  agency 

may  accept  delivery  of  a  product  which  produces  an  unexpected  result  In  a 
hypothesis  test;  the  p-value  and  a  q-value  might  be  used  to  adjust  the  payment 
to  the  supplier.  One  standard  procedure  In  procurement  actions  Is  based  on  a 
Mgld  hypothesis  test,  The  product  Is  rejected  If  the  decision  rule  yields  a 
recommendation  to  reject  the  null  hypothesis  that  the  specification  Is  met. 
There  are  two  problems  with  this  standard  approach.  The  supplier  has  no 

reward  If  the  product  Is  good  enough  to  pass  the  hypothesis  test  with  a 

p-value  higher  than  alpha.  Also,  the  procurer  must  use  additional 


justifications  to  accept  a  product  that  can  be  made  to  satisfy  an  urgent  need 
even  though  It  Is  bad  enough  to  fall  the  hypothesis  test  with  a  q-value  lower 
than  beta.  A  proposed  solution  to  these  two  problems  Is  for  the  procuring 
agency  to  execute  the  following  cost  analysis  procedure: 

(1)  Set  a  specification,  for  the  null  hypothesis,  which  Is  a  value  of  the 
tested  parameter  that  allows  acceptable  but  not  exceptional  operation 
with  the  equlment  or  service, 

(2)  Set  an  alternate  hypothesis  value  of  the  tested  parameter,  as  close 
to  the  specification  as  feasible,  which  Is  not  acceptable. 

(3)  Set  alpha,  beta,  and  the  number  of  measurements  of  the  tested 

parameter  after  balancing  the  risks  and  the  cost  of  testing, 

(4)  Call  for  bids  to  set  the  payment  If  the  hypothesis  test  yields  a 

p-value  equal  to  alpha  and  a  q-valut  equal  to  beta. 

(5)  Establish  a  continuous  scale  of  price  Increases  for  each  p-value  that 
Is  higher  than  alpha. 

(6)  Establish  a  continuous  scale  of  price  decreases  for  each  q-value  that 
Is  higher  than  beta. 

(7)  Obtain  a  random  sample  of  the  equipment  or  service,  perform  the 

hypothesis  test,  and  Implement  the  result  of  the  procedure. 

An  optional  step  might  be  exercised  If  the  supplier  or  the  procuring  agency 
doesn't  accept  the  result: 

(8)  Do  more  testing  If  either  the  supplier  or  procuring  agency  Is  willing 
to  pay  for  It  In  an  attempt  to  raise  the  p-value  or  lower  a  q-valut. 

Naturally,  these  steps  must  be  described  at  the  time  of  the  Invitation  for 
bids  so  potential  bidders  can  decide  how  to  respond.  Steps  5,  6,  and  8 
might  be  topics  for  negotiation  between  the  procuring  agency  and  potential 


suppliers.  This  proposed  procedure  Is  only  one  possible  solution  to 
procurement  confrontations.  Another  procedure  might  be  desired  If  sufficient 
testing  can  establish  that  the  tested  parameter  Is  within  a  narrow  Interval 
with  a  high  degree  of  confidence.  This  7  or  8  step  procedure  should  be 
considered  for  procurements  In  which  limitation  of  testing  Is  expected  to  make 
clear-cut  procurements  decisions  Impractical  or  Impossible. 

EXAMPLE 

The  example  presented  In  Figures  One  through  Eleven  Illustrates 
modification  of  alpha  and  beta  In  a  classical  hypothesis  test  on  the  variance 
of  a  normally  distributed  random  variable.  The  null  hypothesis  HQ  for 
this  test  1$  the  assumption  that  the  variance  Is  less  than  or  equal  to  a 
standard.  The  chi-square  distribution  Is  appropriate  for  the  random  variable 
(n-l)s2/o2  when  n  Is  the  number  of  measurements  used  to  find  the  sample 
variance  s2  which  estimates  the  variance  o2.  The  number  of  degrees  of  freedom 
v  Is  given  by  n-1  for  this  x2  statistic.  The  numbers  In  this  example  have 
been  chosen  to  depict  a  hypothetical  process-improvement  development  In  the 
manufacture  of  glass  with  a  low  variance  In  Its  Index  of  refraction.  Existing 
manufacturing  processes  are  assumed  capable  of  yielding  o2  ■  (10)“*a ;  but  this 
variance  Is  considered  unacceptable  for  the  prototype  process-improvement 
which  Is  Intended  to  reach  a  standard  of  o2  ■  4  (10)“l°. 

Figures  One  through  Four  show  Information  that  Is  useful  before  and  during 
data  taking.  Figures  One  and  Two  contain  the  Information  that  a  statistician 
should  present  to  enable  a  manager  to  complete  the  design  of  the  hypothesis 
test.  Figure  Three  shows  the  designed  hypothesis  test  after  the  manager  has 
selected  a  planned  sample  size.  Figure  Four  reports  the  Information  that  the 


statistician  should  present  to  the  manager  If  pressures  exist  to  change  the 
planned  sample  size. 

Figures  Five  through  Eleven  show  possible  results  for  different  sets  of 
measurements.  In  Figures  Five  through  Eight,  four  different  values  of  sa  are 

used  to  find  the  p-value  and  a  q-value  for  <Ja.  For  any  measurement,  the 

p-value  Is  obtained  by  setting  the  calculated  value  of  the  distributed 
statistic  equal  to  the  statistic  that  yields  a  percentile  given  by  the 
p-value;  that  Is,  alpha  Is  replaced  numerically  by  the  p-value.  The 
calculation  of  a  q-value  Is  similar  to  the  calculation  of  beta  In  that  a  non¬ 
specification  value  of  the  tested  parameter  must  be  used;  the  difference  Is 
that  beta  Is  a  function  of  alpha  while  each  q-value  Is  a  function  of  the 
p-value.  Individual  addition  shows  that  the  four  sets  of  p-values  and 
q-values  In  Figures  Five  through  Eight  have  sums  less  than  one.  Figure  Nine, 
which  Is  not  drawn  to  scale,  Illustrates  graphically  that  the  sum  of  these 
paired  statistics  Is  always  less  than  one.  This  occurs,  for  alpha  and  beta  as 
well  as  for  the  p-value  and  each  associated  q-value,  because  the  area  under 
the  probability  density  function  f < x2 S v )  Is  equal  to  one.  Figure  Nine  also 

shows  that  an  Increase  In  one  of  these  statistics  Is  always  accompanied  by  a 

decrease  In  the  other.  Measurements  leading  to  a  strong  recommendation  to 
reject  H0  will  yield  a  low  p-value  and  high  q-values;  opposite  extremes  In  the 
p-value  and  q-values  result  when  measurements  strongly  Imply  that  H0  should 
iot  be  rejected. 

Use  of  the  p-value  and  associated  q-values  after  the  hypothesis  test,  for 
purposes  other  than  gaging  the  Intensity  of  the  decision  to  reject  or  not 
reject  H0,  requires  analysis  curves.  Some  of  these  curves  are  presented  and 
Interpreted  In  Figures  Five,  Six,  Ten,  and  Eleven.  In  Figures  Five  and  Six 
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where  s2  Is  close  to  the  critical  sample  variance  s2  which  separates 
rejection  and  non-rejection  of  the  null  hypothesis,  power  curves  are 
presented  for  the  manager  to  use  In  considering  additional  testing.  Also  In 
Figures  Five  and  Six,  hypothetical  algorithms  are  presented  for  adjusting  the 
cost  and  the  results  of  these  algorithms,  the  p-value,  and  a  q-value  for  o2 
are  given.  It  must  be  emphasized  that  the  manager,  In  consultation  with  the 
statistician,  should  establish  these  cost  adjustment  algorithms.  The  manager 
may  design  an  algorithm  other  than  a  straight  line;  he/she  may  also  use  a 
maximum  p-value  pm  and  a  maximum  q-value  qm  other  than  the  simple  ones  used 
In  Figures  Five  and  Six  to  limit  the  change  In  cost  caused  by  results  of  the 
hypothesis  test.  Figure  Ten  summarizes  the  numbers  of  Figures  Five 
through  Eight  which  are  all  based  on  a  sample  size  equal  to  that  planned  In 
Figure  Three.  Figure  Eleven  presents  a  similar  summary  for  slightly  different 
actual  samples  sizes, 

CONCLUSION 

Modification  of  alpha  and  beta  makes  hypothesis  testing  less  rigorous  and 
allows  managers  more  flexibility.  Modification  of  beta  In  the  Interim  between 
the  design  of  the  statistical  test  and  the  completion  of  physical  measurements 
allows  managers  to  make  Informed  decisions  concerning  changes  to  the  proposed 
number  of  physical  measurements.  Modification  of  alpha  and  beta  after  the 
hypothesis  test  Is  completed  allows  managers  to  make  Informed  decisions 
about  (1)  consolidating  the  result  of  the  decision  rule  with  factors  not 
considered  by  the  hypothesis  test,  (2)  the  value  of  additional  testing,  and 
(3)  compensation  when  specifications  can  be  considered  as  met  with  risks 
different  from  the  original  alpha  and  beta. 


Figure  One  —  Partially  Designed  Hypothesis  Test: 


Part  1  — 


H0:  o*<a*Q  .-4(l0)-i® 


Ha:  o2  >  o2 


a  ■  P  [Reject  H0  |  ■  .01 

Test  Is  One  Tailed 


Part  2  —  Rejection  Region: 


f(x*;v) 


X2  >  x£,v  “  Xa.n-1 

where  x2  ■  (n*l)s2/o2 

with  n  ■  number  of  measurements 

and  a2  ■  sample  variance 


Rejection  Measurement.  x|  v  ■  (n-1 ) *c/°S  *c  *  °o  x2in-i  t  (n-D 


.1 


I 


Figure  Three  —  Hypothesis  Designed  with  n*»7: 


Part  1  —  H0:  o2  j<  c2  ■  4(10)"10 


Hft:  °2  >  ®o 


Part  2  —  Rejection  Region: 


f(x2;v) 


a  ■  P  [Reject  H0  |  a2]  ■  .01 


Test  Is  One  Tailed 


x2  >  x2,v  -  X20in-i  -  x2 ,oi,6  "  l6*8 
where  x2  ■  (n-l)s2/o£  ■  6s2/o£ 
with  n  «  number  of  measurements 
and  s2  ■  sample  variance 


Rejection  Measurement:  x2  ■  xJ|V  ...>  ,2  .  02  X2Q1  6/6  .  u.2(io)-3l< 


B  Part  —  If  fl}"(10)~9!  9  «  Pt$2<s|3»Pt(n«l)s2/o5<{(n-l)}/oJ}s|]"P[x2<.672]  *  .005 


FlgurG  Four  —  9T  and  Power  •  irT  •  1  9T  Curves  If  n  is  Changed  to  3: 


$T  Part  —  If  aj-(10) ”8 ;  $T  ■  P[s2<s|3  -  P[xa<{e§^°uJ*S,n-l^  “  ptx2<.368]  «  .168 


Power 


7 

3  6 


o2(10)8 


?2(10 )® 


Figure  Five  —  Inferential  Information  from  Design  and  Hypothesis  Test  If  s2>a2 
and  s2  Implies  Rejection  of  H0:  o2  ±  o2  ■  AdO)”10  with  a«.01: 


Part  3  — -  If  s2  ■  12.3(10)"10  from  n«7  measurements;  x2  «  (n-l)s2/o2  ■  18.45 

Part  4  —  Since  x2  "  18.45  >  16.81  ■  x20i,fi  ■  x^-i  m  x|,vi  Reject  H0 

p  Part  —  To  Barely  Not  Reject  Hqj  Xp|V  ■  Xptg  ■  18.45  ■••*>  p-value  *  .005 

q  Part  —  If  a2  ■  (10)“8;  q-value  ■  l-irq  *  P[x2<{e|/02}XptV3  *  PCx2<.H]  ■  .006 


Power  Curves: 


Price  Decrease 
In  Percentage 
of  Target  Cost 


Use  of  the  q-value  of  .006  and 
the  graph  to  the  left  yields  a 
price  reduction  equal  to  .017* 
of  the  target  cost 


B  ■  ,005  from  design 


%  •  P[x2<{o2/c2}x2|V  |  XptV-(n-l)s2/oJ  and  s2«o2J  ■  P[xz<(n-1)]  -  PC x2 <63  ■  .58 


Figure  Six  —  Inferential  Information  from  Designed  Hypothesis  Test  if  s2>o|  and 
s2  Does  Not  Imply  Rejection  of  Hp:  o2  ±  o2  „  4( io)-10  with  a-. 01: 


Part  3  —  If  s2  «  8.4(10)“10  from  n*7  measurements;  x2  •  (n-l)s2/o2  ■  12.6 
Part  4  —  Since  x2  ■  12.6  <  16.81  ■  X2qi,6  *  xa.n-l  *  xakv*  00  Not  ^eJect  Ho 


p  Part  —  To  Barely  Reject  H0;  ■  x2Pte  *  *2*6  P“Va1ue  -  .06 

q  Part  —  If  o2  -  (in)*8;  q-value  ■  l-wq  *  P[x2<{02/«u}x$#v2  *  P[<2<.50]  -  .002 


Power  Curves: 


.2 


& 


Price  Increase 
In  Percentage 
of  Target  Cost 


10 


Use  of  the  p-vslue  of  .06  and 
the  graph  to  the  left  yields 
a  price  enhancement  equal  to 
.17*  of  the  target  cost 


a  -  .01  from  design 


Pm  ■  Ptx2  >  x2  v3  ■  PCx2  >  (n-l)s2/oJ]  »  Pl!x2  >  ■  PCx2  >  6]  -  .42 

^  Pm*  ® 
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Figure  Sevin  —  Inferential  Information  from  Designed  Hypothesis  Test  If  s2  >o2u 
So  s2  Forces  Management  to  Use  a  Quite  Low  p-value  to  View  the 
Test  of  H0:  o2  ±  dj,  -  4(10)“10  with  a  «.01  as  Indecisive: 

Part  3  —  if  s2  ■  22(10 )“10  from  n*7  measurements $  x2  •  (n-l)s2/o2  ■  33 

Pert  4  — -  Since  x2  -  33  >  16.81  -  X2oi,s  *  4#n-l  "  R*Ject  Ho  !;• 

p  Par't  —  To  Barely  Hot  Reject  H0i  Xp>v  *  Xp,g  *  33  p-value  ■  .00001 

q  Part  —  If  o2  -  (10)"*;  q-value  ■  l-flq  PC^<{q|/q^x|ty3  -  Pty.2 <1.32]  ■»  .03 

Figure  Eight  —  Inferential  Information  from  Designed  Hypothesis  Test  If  fc2  ± 

So  s2  Forces  Management  to  Use  a  Quite  High  p-value  to  View  the 
Test  of  H0:  o2  .<  ■  4(10)~l°  with  a  ■  .01  as  Indecisive: 


part  3  ...  if  ,s*  .  2.85  (10 )”10  from  n*7  measurements;  x2  ■  (n-l)s2/o2  ■  3.8*5 
Part  4  —  Since  x2  ■  3.825  <  16.81  -  x20i,6  *  4,n-l  “  4,v;  Do  Mot  R*Ject  Ho 
p  Part  — -  To  Barely  Reject  HQ.  xJ|V  *  X^g  .  3,825  — ■>  p-value  ■  .700 

q  Part  —  If  o2  »  (10)“8;  q-value  11  PCx2<{ *  P^X2 <-1533  “  ♦QQ007 
Figure  Nine  —  a  4  0  anj  p  a  q  on  Graphs  of  s2  vs  f(x2;v)  with  v-n-1  4  x2-vs2/o2: 
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Figure  Ten  —  Summary  of  Results  for  Representative  Values  of  s2  from 

n  ■  7.  Measurements  to  Test  IT  :  o2  <  o2  ■  4(10)-10  wt 

o  o 

*  ■  .01.  x*  ■  16.81,  and  $  ■  .005  for  o2  ■  (10 )“8 : 

v*  |il  X 


■m 
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s2: 


X2  «  (n-l)  s2/e2: 


Decision  on  H0i 


22(10)“10  12.3(10)“10 


33  >  16.81  18.45  >  16.81 


Reject 


Reject 


p-value: 


,00001c. 01»a  ,005<.01«a 


q-value  for  o2:  .03>,005"6  .OO6>,OOS"0 


One  Resultant 
Change  In  Cost: 


Minus  ,4X  Minus  .017X 


8.4(10)-10 
12.6  >  16.31 
Do  Not  Reject 
,05>.01"o 
,002<, 005-8 
Plus  IX 
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Figure  Eleven  Summary  of  Results  for  Representative  Values  of  r,2  from 
n  ■  -|g  Measurements  to  Test  H^:  o2  £  o2  *  4(lO)“*10  with 

,01»  xa,n-l  "  {loioS  *  and  *  “  {io008  for  °u  "  (10)“8; 
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ABSTRACT  In  April  and  May,  1084,  The  field  Artillery  Board,  Ft  Sill,  OK  conducted 
a  Force  Development  Test  and  Experimentation  (FDT&E)  of  the  Fire  Support  Team 
(FIST)  concept  at  Ft.  Riley,  KS.  The  purpose  of  the  FDT&E  was  to  test  and  evaluate 
the  effectiveness  of  the  FIST  HQ  equipped  with  FIST  vehicles  and  digital  communica¬ 
tions  equipment  under  various  tactical  configurations,  selected  modes  of  operation  end 
personnel  shortages.  Although  traditional  manual  data  collection  methods  employing 
human  observers  was  used  to  record  test  data,  a  new  ..utomatic  data  recording  tech¬ 
nique  based  on  the  Artillery  Control  Environment  (ACE)  technology  was  used  for  the 
first  time  in  the  field.  Personnel  from  the  Ballistic  Research  Laboratory  (BRL)  assisted 
in  the  experimental  desigu,  and  were  responsible  for  the  designing,  coding  and  testing 
the  computer  software  for  the  data  collection  and  reduction  system* 

The  discussions  will  focus  ou  the  experimental  design,  data  reduction  methodology, 
the  methods  of  analysis  employed  and  a  brief  reporting  of  the  results. 


A.  Background 

During  April  and  May  1084,  The  Field  Artillery  Board,  Ft.  Sill,  OK  conducted  a 
Force  Development  Testing  and  Experimentation  (FDT&E)  of  the  Fire  Support  Team 
Headquarters  (FIST  HQ)  concept  at  Ft.  Riley,  KS.  The  test  consisted  of  three  iterations 
of  a  120-hour  Scenario  Oriented  Recurring  Evaluation  System  (SCORES)  field  exercise 
that  was  based  upon  and  included  the  mechanized  infantry  and  armor  defensive 
maneuvers.  The  task  force  was  confronted  by  an  opposing  force  (OPFOR)  of  various 
strengths  and  a  jamming  team.  All  elements  were  strictly  controlled  by  tl’ife'  test 
directorate  during  the  first  two  exercises.  The  tliird  exercise  was  a  frecplay, 
uncontrolled  force  on  force  exercise. 

Personnel  from  tho  Ballistic  Research  Laboratory  (BRL)  designed  the  experiment 
and  assisted  in  the  implementation  of  experimental  design  methodology  in  the  controlled 
segment  of  the  test  to  address  a  subset  of  the  overall  objectives.  In  addition,  personnel 
from  the  BRL  were  responsible  for  designing,  coding  and  testing  a  new  automatic  data 
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recording  and  reduction  system  based  on  the  Artillery  Control  Environment  (ACE) 
technology.  This  report  will  focus  on  the  experimental  design,  data  reduction  and 
recording  methodology,  the  methods  of  analysis  employed  and  a  brief  discussion  of  the 
results.  The  analysis  for  this  report  is  based  upon  data  obtained  from  the  Field 
Artillery  Board,  Ft,  Sill,  OK. 

B.  Purpose 

The  overall  purpose  of  the  FDT&E  was  to  evaluate  the  operational  effectiveness  of 
the  FIST  HQ  equipped  with  a  Fire  Support  Team  vehicle  (FIST  V)  and  digital 
communications.  Test  results  will  be  used  by  the  United  States  Army  Field  Artillery 
School  (USAFAS)  to  furthor  develop  FIST  operational  and  organizational  concepts. 

To  demonstrate  this  effectiveness,  a  study  of  the  FIST  HQ  ability  to  perform  fire 
support  coordination  under  two  modes  of  Forward  Observer  (FO)  message  control  and 
four  types  of  FIST  HQ  configuration,  while  under  various  workloads,  was  conducted. 


1L.-TESI  CQM.CEKX 

A.  Objectives 

1)  To  determine  whether  or  not  FIST  DMD  message  control  of  the  FO’s,  in  the 
review  and  automatic  communication  modes  of  operation,  has  an  effect  on  the 
FIST  HQ  ability  to  perform  Art*  support  coordination, 

2)  To  determine  If  the  FIST  HQ  can  perform  firo  support  coordination:  a)  with 
the  Ground/Vehicle  Lyier  Location  Designator  (G/VLLD)  mounted  on  the  FIST 
V  with  all  FIST  personnel  present,  b)  with  the  G/VLLD  mounted  without  the 
FIST  Chief,  c)  inside  the  FIST  V  in  a  Buttoned-up  environment  and  d)  >vith  the 
G/VLLD  dismounted  from  the  FIST  V.  To  dismount  the  G/VLLD  from  the  FIST 
V,  two  FIST  HQ  personnel  must  dismount  both  the  G/VLLD  and  its  associated 
equipment. 

3)  To  determine  if  mission  workload  affects  the  FIST  HQ  performance  of  fire 
support  coordination,  Mission  workload  was  defined  as  the  number  and  types  of 
missions  the  FIST  HQ  was  required  to  process  simultaneously.  There  were  four 
fire  mission  types:  missions  initiated  from  the  mechanized  infantry  FO,  missions 
initiated  from  the  armor  platoon  leader  (Armor)  by  voice,  FIST  HQ  shooting 
COPPERHEAD  munitions,  and  missions  from  the  FIST  HQ  shooting 
conventional  munitions. 

B.  Measure  of  Performance 

A  measure  of  performance  (MOP)  Is  a  response  that  is  used  to  quantify  the  effects 
of  the  factors  to  be  evaluated.  For  FIST  initiated  missions,  it  was  defined  as  the  elapsed 
time  from  target  acquisition  until  the  (fire  request)  message  b  transmitted  from  the 
FIST  Digital  Message  Device  (DMD).  Service  time  for  armor  missions  was  the  time 
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from  receipt  of  a  fire  request  message  by  the  FIST  HQ  until  transmission.  The  time  to 
service  FO  missions  was  the  elapsed  time  from  when  the  acknowledgment  (ACK)  is  sent 
from  the  FIST  DMD  indicating  receipt  of  a  fire  request  message  until  the  message  is 
transmitted.  This  measure  indicates  the  combined  time  a  message  spends  in  the  FIST 
DMD  message  queue,  and  the  processing  and  decision  time  of  the  FIST  HQ. 

C.  Scope 

The  first  two  field  exercises  (FEX  1,  FEX  2),  which  were  a  combination  of  Live  Firo 
and  Force  on  Force,  utilized  three  FIST  HQ  and  one  combat  observation  lasing  team 
(COLT)  attached  to  a  mechanized  infantry  task  force  that  consisted  of  two  mechanized 
infantry  companies  and  one  armor  company.  (See  Figure  1). 

The  FIST  HQ  consisted  of: 

1.  The  Fire  Support  Team  Chief 

2.  The  Fire  Support  Team  Sergeant 

3.  Two  radio  telephone  operators 

All  members  of  the  FIST  HQ  were  trained  in  the  operation  of  the  FIST  DMD.  Nine 
weeks  of  individual  training  was  conducted  and  validated  by  the  USAFAS.  This 
individual  training  was  followed  by  two  weeks  of  collective  training. 

D.  Limitations 

After  receiving  the  initial  fire  request  message  from  a  FO  and  deciding  hoSv  the 
fire  request  should  be  handled,  the  FIST  HQ  routed  all  subsequent  messages  for 
that  fire  mission  through  the  FIST  DMD  in  the  automatic  "mission  mode."  That 
is,  all  subsequent  messages  for  that  fire  mission  were  automatically  routed 
through  the  FIST  DMD.  Operator  intervention  was  needed  only  if  a  message  did 
not  get  acknowledged  in  four  transmissions. 

2)  Electronic  Warfare  was  prohibited  during  the  controlled  portion  of  the 
FDT&E. 

3)  Range  regulations  at  Fort  Riley  prevented  the  G/VLLD  from  being  employed 
in  a  totally  realistic  environment.  Laser  designation  and  range  finding  were 
allowed  in  only  two  locations  and  even  then  had  to  be  restricted. 

4)  The  control  cells  that  contained  the  Buttoued-up  configuration  were  run  at 
night. 


COMPANY  FIRE  CONTROL  NET  (CFC) 

BATTALION  MORTAR  FIRE  DIRECTION  NET  (BN  MTR) 
FIELD  ARTILLERY  COMMAND  FIRE  NETS  (CHI ,  CF2) 
FIELD  ARTILLERY  FIRE  DIRECTION  NETS  (FD1  FD2) 
COMPANY  COMMAND  NET  (CO  CMD) 

TASK  FORCE  COMMAND  NET  (TF  CMD) 

NOTE:  THE  TF  WAS  COMPOSED  OF  THREE  SIMILARLY 
ORGANIZED  COMPANY  TEAMS 


Figure  1.  Fire  Support  Structure. 
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E.  Data  Collection 


In  addition  to  manual  data  collection  methods  employing  human  observers  to 
record  test  data,  a  new  automatic  data  recording  technique  based  on  the  ACE 
technology  was  used  for  the  first  time  in  a  field  exercise.  The  procedure  consisted  of 
recording  digital  radio  traffic  time  coded  on  analog  magnetic  tape.  Every  24  hours  the 
tapes  were  shipped  to  Aberdeen  Proving  Ground  (APG),  MD  where  HEL  personnel 
received  the  tapes  an'd  played  them  back  into  the  computer  controlled  message 
collection  and  reduction  system.  The  resulting  sorted  list  of  messages  was  then  written 
to  digital  magnetic  tapes  and  shippod  to  Ft.  Sill  for  analysis. 


UL  MESSAGE  COLLECTION  AND  REDUCTION  SYSTEM  The  major  components  of 
the  message  collection  and  roduction  system  were 

1)  Bit  Boxes  (Tactical  Communication  Modems,  TCM) 

2)  VAX  11/750  Computer 

3)  BRL  VAX  Unix  Operating  System 

4)  Message  collection  and  reduction  software 

A.  Hardware 

Bit  Boxes  are  microprocessor  based  modems  which  enable  Tactical  Fire  Direction 
System  (TACF1RE)  hardware  to  communicate  with  commercial  computers.  The  Hit 
Boxes  convert  Frequency  Shift  Keyed  (FSK)  variable  format  and  fixed  format  TAGF1IM3 
messages  (from  wire  line  or  radio)  to  RS232  ASCII  character  format  which  commercial 
computers  can  accept,  and  visa  versa. 

A  DEC  VAX  11/750  computer  was  available  for  use  as  the  main  computer  to 
support  the  message  collection  and  reduction  software.  The  computer  operating  system 
was  a  BRL  enhanced  version  of  4.2  BSD  (Berkley  System  Distribution)  Unix. 

B.  Software 

The  application  software,  which  was  written  in  the  C  programming  language,  had 
two  primary  tasks:  1 )  message  collection,  and  2)  message  roduction. 

The  message  collection  program  receives  streams  of  characters  from  the  Bit  Boxes, 
separates  the  streams  into  complete  messages,  records  the  start  and  end  time  of  each 
message,  and  stores  this  information  in  a  computer  file. 

The  data  reduction  program  reads  the  data  files  created  by  the  message  collection 
program.  The  purpose  of  this  program  is  to  sort  the  messages  into  fire  missions,  The 
result  is  3  other  files  that  contain  (1)  a  list  of  messages  categorized  by  fire  mission  target, 
number,  (2)  a  list  of  messages  believed  to  be  associated  with  a  fire  mission  but  for  some 
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reason  that  mission  could  not  be  identified,  (3)  and  a  list  of  messages  that  are  known 
but  not  part  of  a  fire  mission.  These  lists  of  messages  were  shipped  to  Ft.  Sill  and 
combined  with  manual  data  to  create  a  comprehensive  data  base  for  analysis.  For  au 
indepth  description  of  the  message  collection  and  reduction  system  see  "Field  Artillery 
Digital  Message  Collection  and  Reduction  Software,"  BRL-IMR-822,  June  1684. 


IV.  EXPERIMENTAL  DESIGN 


A.  Factors 

The  three  factors  that  were  tested  during  the  controlled  portion  of  the  FDT&E 
were  FIST  Configuration,  Mode  of  FIST  DMD  Control  and  Mission  Workload. 

1)  FIST  Employment  Configuration  alternatives  were: 

a)  G/VLLD  mounted  •  all  hatches  on  the  FIST  V  were  open  and  the 
C/VLLD  was  mounted  with  the  entire  FIST  HQ  present. 

b)  G/VLLD  mounted  without  FIST  Chief-  all  hatches  on  the  FIST  V  wore 
open  and  the  G/VLLD  wm  mounted  with  the  FIST  Chief  not  available. 

c)  G/VLLD  dismounted  -  the  G/VLLD  was  placed  away  from  the  vehicle 
along  with  two  of  the  four  FIST  HQ  members. 

t 

d)  Buttoned-Up  •  all  hatohes  on  the  FIST  V  were  dosed  and  the  G/VLLD 
was  mounted  with  the  entire  FIST  HQ  present. 

2)  Mode  of  FIST  DMD  Control 

a)  Review  •  FIST  DMD  stops  all  initial  fire  request  messages  from  platoon 
FO’s  for  the  FIST  HQ  to  review. 

b)  Automatic  •  FIST  DMD  immediately  forwards  all  initial  fire  request 
messages  with  out  action  by  the  FIST  HQ. 

3)  Mission  Workload 

Mission  workload  was  defined  as  the  number  and  types  of  fire  missions  the 
FIST  HQ  were  required  to  process  simultaneously.  The  four  types  of  fire 
missioni'1  were: 

1)  CONV  *  FIST  HQ  shooting  a  conventional  munition 

2)  CPH  -  FIST  HQ  shooting  a  COPPERHEAD  munition 

3)  ARMOR  -  Missions  initiated  by  the  armor  platoon  leader  and 
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received  by  voice  at  the  FIST  IIQ. 

4)  FO  -  Missions  initiated  by  the  mechanized  infantry  FO  and 
transmitted  digitally  to  the  FIST  HQ. 

Based  on  seven  combinations  of  mission  types,  thirteen  categories  of 
mission  workloads  were  deQned.  They  are: 

TABLE  I.  MISSION  WORKLOAD 


CATEGORY 

FIRE  MISSIONS 

PROCESSED  SIMULTANEOUSLY 

MISSION 

TYPE 

a  i 

1CPH 

CPU 

b 

1F0  4  1C0NV 

CONV 

c 

1  FO  4  1  CONV 

FO 

d 

1  ARMOR  4  1  CONV 

CONV 

e 

1  ARMOR  4  1  CONV 

ARMOR 

f 

1  FO  4  1  CPH 

CPH 

S 

1  FO  +  1  CPH 

FO 

h 

1  FO  4  1  ARMOR  4  1  CPH 

CPH 

i 

1  FO  4  1  ARMOR  +  1  CPH 

ARMOR 

j 

1  FO  4  1  ARMOR  4  I  CPH 

FO 

k 

2  FOs 

FO 

1 

1  ARMOR  4  2  FOs 

FO 

m 

1  ARMOR  4  2  FOs 

ARMOR 

B.  Design  Matrix 

It  was  decided  that  the  smallest  period  of  time  reasonable  to  test  any  one  of  the 
treatment  combinations  was  two  hours.  A  factorial  design  was  constructed  with  each 
experimental  combination  being  tested  in  a  random  order.  This  scheme  assured  that  the 
effect  of  each  of  the  experimental  combinations  on  the  FIST  HQ  ability  to  perforin  fire 
support  coordination  could  be  measured.  The  FIST  HQ  were  tested  under  all  of  the 
experimental  combinations  and  the  design  was  repeated  for  each  of  the  two  controlled 
iterations  of  the  FDT&E.  The  design  matrix  is  presented  in  Table  2. 


V.  STATISTICAL  ANALYSIS  The  analysis  for  this  section  is  based  on  data  reduced  by 
the  Field  Artillery  Board,  Ft.  Sill,  OK,  which  wa*  a  combination  of  manual  data 
collected  by  human  observers  and  digital  data  that  was  sorted  by  the  message  collection 
and  reduction  system.  This  section  is  intended  to  be  a  supplement  to  the  data  analysis 
conducted  by  the  Field  Artillery  Board  and  only  focuses  on  several  key  factors  and  their 
associated  levels.  Unfortunately,  the  Buttoned-up  level  of  the  FIST  Employment 
Configuration  factor  was  not  available  in  this  subset  of  the  FDT&E  data  base,  but  will 
be  analyzed  in  a  future  BRL  report. 
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A.  Transformation 

As  the  data  was  being  checked  for  completeness,  it  was  noted  that  the  distribution 
of  service  time  was  skewed  and  that  the  variances  of  the  observations  under  various 
experimental  conditions  were  different.  Further  investigation  of  the  data  revealed  a 
positive  correlation  between  the  cell  standard  deviations  and  the  cell  means. 
Correlation  between  the  standard  deviations  and  cell  means  is  often  accompanied  by 
marked  non-normality  and  non-homogeneity  of  variance,  and  indicates  that  the 
particular  form  of  the  original  observations  is  unsuitable  for  Analysis  of  Variance 
(ANOVA)  procedures. 

However,  a  transformation  can  be  determined  which  makes  the  standard  deviation 
independent  of  the  mean,  corrects  non-homogeneity  and  also  results  in  the  observations 
being  distributed  more  normally.  In  general,  if  a  significant  functional  relationship 
between  the  standard  deviations  and  the  group  means  can  be  determined,  then  the 
transformation  is  the  iutegral  of  the  reciprocal  of  this  functional  relationship.  Using  this 
procedure,  the  following  transformation  was  developed: 

i 

1.7  In  (lfi.9  +  .56  (service  time)  ) 


The  transformed  data  became  more  normal  and  the  homogeneity  of  variance 
among  the  experimental  conditions  was  improved. 

B.  Analysis  Of  Variance 

An  analysis  of  variance  procedure  was  performed  on  the  transformed  data  with  one 
slight  modification  to  this  procedure  due  to  unequal  experimental  group  sizes,  The  sum 
of  squares  for  all  terms  in  the  model,  except  the  error  term,  was  weighted  by  the 
harmonic  mean.  The  ANOVA  is  presented  in  Table  3.  A  star  next  to  the  F-ratio 
Indicates  the  factor  is  significant  at  the  alpha  level  of  .05.  Since  this  analysis  assumes  a 
fixed  effects  model,  the  denominator  for  all  F-ratios  is  the  pooled  error  term. 

Since  the  ANOVA  was  performed  on  the  transformed  data,  it  was  decided  that 
comparisons  of  medians,  calculated  on  observed  service  times,  would  be  more 
meaningful  than  comparing  transformed  means. 

C.  Results 

The  most  significant  term  in  the  analysis  was  mission  workload.  One  reason  for 
this  significance  is  that  it  took  substantially  loss  timr  to  service  fire  request  messages 
from  mechanized  infantry  FO  missions  than  either  the  FIST  HQ  missions 
(COPPERHEAD  or  Conventional)  or  Armor  missions.  In  both  FIST  DMD  control 
modes,  the  FIST  HQ  initiated  fire  request  messages  require  data  input,  review,  and 
transmittal.  Armor  messages,  which  are  received  by  voice,  must  be  reviewed  and  input 
os  digital  messages  by  the  FIST  HQ;  whereas  the  digital  FO  fire  requests  require  only 
review  and  transmittal  in  the  review  mode  of  FO  control  and  no  processing  at  all  in  the 
automatic  mode. 


TABLE  3.  ANALYSIS  OF  VARIANCE 
(SERVICE  TIME) 


SOURCE 

DEGREES  OF 
FREEDOM 

SUM  OF 
SQUARES 

MEAN 

F 

RATIO 

10.60* 

Mission  Workload 

12 

■m 

8.42 

Mode 

1 

5.65 

5.65 

7.14* 

ConAguration 

2 

0.025 

■■ 

m 

Mission  Workload  x 
Mode 

12 

0.21 

0.77 

<  i 

Mission  Workload  x 
ConAguration 

24 

13.43 

0.56 

<  i 

ConAguration  x 
Mode 

2 

0.08 

0.04 

<  i 

Mission  Workload  x 
Mode  x  ConAguration 

24 

12.33 

0.51 

<  i 

Pooled  Error 

461 

wnmm 

0.70 

Another  interesting  result  observed  was  that  in  mission  combinations  in  which 
Armor  missions  were  processed,  Armor  missions  had  a  longer  service  time  than  any 
other  mission  type.  This  trend  seems  to  indicate  that  it  takes  the  FIST  HQ  longer  to 
process  voice  initiated  fire  request  messages  than  to  initiate  his  own  or  service  FO 
missions.  This  result  is  not  surprising  since  it  takes  longer  to  input  a  message  manually 
than  to  receive  one  digitally.  These  trends  were  consistent  in  both  the  Automatic  and 
Review  modes  as  shown  in  Table  4. 

The  number  of  missions  processed  simultaneously  also  affected  FIST  HQ  service 
time.  In  plotting  the  median  service  time  for  the  mechanized  infantry  FO  fire  missions 
in  review  mode  (See  Figure  2),  one  can  see  that  it  takes  the  FIST  HQ  longer  to  service 
FO  missions  when  the  FIST  HQ  are  also  initiating  COPPERHEAD  missions  and 
receiving  a  armor  message  than  when  the  FIST  HQ  ai~  just  servicing  FO  missions  and 
shooting  COPPERHEAD.  In  addition,  the  FIST  IIQ  service  time  for  FO  Are  request 
messages  is  shorter  when  they  are  also  initiating  a  conventional  mission  as  opposed  to 
also  shooting  COPPERHEAD.  This  result  is  not  surprising.  When  the  FIST  HQ  is 
initiating  a  COPPERHEAD  mission  while  in  the  review  mode,  the  FIST  DMD  operator 
functions  are  disabled  after  sending  a  FO  Command  (Fire)  or  a  Fire  Request  Quick 
Message  and  no  action  can  be  taken  by  the  FIST  DMD  operator  until  the  X  button  is 
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Figure  2.  Mechanized  Infantry  PO  Initiated  Messages  By  Workload. 


TABLE  4.  MISSION  WORKLOAD  BY  MODE 
(MEDIAN  SERVICE  TIME) 
(SECONDS) 


Mode 


MISSION  WORKLOAD 


CPH 


CPU 


ONV 


CONV 


FO 


CPH 

55 

FO  +  CPH 

11 

FO  +  ARMOR  +  CPH 

6 

FO  +  CONV 

88 

ARMOR  +  CONV 

77 

FO  +  CONV 

10 

FO  +  CPII 

It 

FO  +  ARMOR  +  CPH 

20 

2  FOs 

22 

2  FOs  +  ARMOR 

5 

ARMOR  +  CONV 

79 

FO  +  ARMOR  +  CPH 

81 

ARMOR  +  2  FOs 

58 

FO 


FO 


ARMOP 


ARMOR 


ARMOR 


pressed  to  end  the  COPPERHEAD  mission.  Surprisingly,  the  FIST  HQ  spent  the 
longest  time  servicing  mechanized  infantry  FO  missions  m  review  mode,  when  they  were 
not  initiating  or  reviewing  any  other  mission  types.  In  this  mission  workload  (2~FOs), 
the  FIST  HQ  only  responsibility  was  to  review  the  two  messages  received  from  his 
mechanized  infantry  FOs.  FIST  personnel  spent  a  lot  of  time  reviewing,  changing,  and 
deciding  if  the  initial  lire  request  message  should  be  sent  to  TACF1RE  or  to  one  of  their 
local  resources,  such  as  the  battalion  mortar  platoon. 
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From  Figure  3,  one  tan  see  that  it  took  more  time  for  the  FIST  HQ  to  service 
armor  messages  when  they  were  also  shooting  a  COPPERHEAD  or  conventional  mission 
than  when  they  were  only  reviewing  a  mechanized  infantry  FO  messages  and  serving 
armor  missions.  This  trend  is  consistent  with  both  the  automatic  and  review  FIST 
DMD  control  mode. 

In  the  automatic  control  mode,  all  initial  fire  request  messages  received  by  the  FIST 
HQ  are  automatically  forwarded  to  their  destination.  Initiating  messages  in  the  review 
mode  must  be  passed  by  the  FIST  DMD  operator  before  they  can  be  transmitted. 
Therefore,  one  could  expect  the  FIST  DMD  mode  of  control  to  significantly  affect  the 
time  it  takes  to  service  digital  fire  request  messages.  The  ANOVA  table  revealed  that 
the  Mode  of  FIST  DMD  Control  was  significant. 

The  percent  of  all  messages  processed  by  service  time  in  the  automatic  and  review 
modes  are  shown  in  Figures  4  and  5,  respectively.  The  median  service  time  for  the 
automatic  mode  was  small,  7.0  seconds,  when  compared  to  the  median  service  time  of 
20.0  seconds  to  service  messages  in  the  review  mode.  For  mechanized  infantry  FO 
missions,  the  median  service  time  in  the  review  mode  ranged  between  6.0  and  22.0 
seconds  over  all  workloads.  However,  ip  the  automatic  mode,  the  median  service  time 
for  all  workloads  was  2.0  seconds.  This  trend  was  not  as  prevalent  for  messages 
initiated  by  the  FIST  HQ  or  messages  received  by  voice  from  the  armor  as  depicted  in 
Table  4. 

It  is  worth  noting  that  .  FIST  Employment  Configuration  was  not  statistically 
significant.  The  FIST's  ability  to  service  FO  and  ARMOR  missions  as  well  as  initiate 
his  own  missions  was  not  affected  by  the  various  configurations.  This  implies  that  the 
FIST  can  perform  efficient  fire  support  coordination  when  the  FIST  Chief  is  not  present 
or  when  two  members  of  the  FIST  are  not  available  (due  to  the  G/VLLD  being 
dismounted).  However,  this  infers  nothing  about  the  quality  of  the  decision  being  made. 

One  puzzling  result  was  that  the  median  service  time  for  a  FIST  HQ  to  service 
COPPERHEAD  missions  wnile  in  review  mode  and  for  mission  workload  (FO  + 
ARMOR  4-  CPH)  was  only  6.0  seconds  and  only  11.0  seconds  for  workload  (FIST  FO  + 
CPH).  Looking  at  the  service  time  distribution  for  these  two  categories,  one  notes  a 
bimodal  distribution  which  may  indicate  the  presence  of  a  lurking  variable. 

Cluster  analysis  was  used  to  try  to  categorize  the  COPPERHEAD  missions  into 
two  groups:  This  is  a  multivariate  statistical  technique  in  which  COPPERHEAD 
missions  were  separated  into  groups  based  on  the  minimization  of  variance  within 
groups  and  the  minimization  of  the  distance  between  groups.  A  difference  in  values 
among  groups  from  different  COPPERHEAD  mission^  is  said  to  exist  if  the  hypothesis 
of  equality  of  means  among  groups  is  rejected  by  an  F-test  with  a  significance  level  of 
.05.  The  number  of  groups  in  which  to  categorize  the  COPPERHEAD  missions  was  not 
specified. 

Using  cluster  analysis  on  the  COPPERHEAD  mission  service  time,  two  populations 
were  identified.  One  group  had  a  median  service  time  of  8.0  seconds  and  a  range 
between  1.0  and  32.0  seconds.  The  other  group  centered  at  56.0  seconds  and  ranged 
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Pig’iru  5,  Percent  of  Messages  Processed  By  Service  Time  in  Review  Mode. 


between  34.0  and  100,0  seconds.  The  groups  were  statistically  different  at  a  significance 
level  of  .OS.  Based  on  the  above  categorization  scheme,  the  analysis  was  redone  with  the 
two  types  of  COPPERHEAD  missions.  This  resulted  in  workload  having  sixteen 
categories.  The  conclusions  remained  unchanged  from  the  original  analysis.  Mission 
Workload  and  Mode  of  FIST  DMD  Control  were  the  only  factors  determined  to 
significantly  affect  FIST  service  time  especially  in  regard  to  FO  missions.  The  median 
service  time  for  the  two  groups  of  COPPERHEAD  missions  by  Mode  of  FIST  DMD 
Control  and  Mission  Workload  are  given  in  Table  5.  The  median  service  time  for  the 
group  with  the  smaller  median  service  time  ranged  between  6.0  and  9.0  seconds  while 
the  second  group  ranged  between  4S.0  and  92.0  seconds.  No  statistical  differences  were 
found  between  the  review  and  automatic  modes  of  FIST  DMD  Control  for  either  group. 
Similarly,  Mission  Workload  had  no  effect  on  either  category  of  COPPERHEAD 
missions. 

There  are  several  possible  reasons  as  to  why  there  are  two  categories  of 
COPPERHEAD  mission  service  time.  One  reason  is  that  terrain  conditions  will  strongly 
influence  COPPERHEAD  service  time.  Another  reason  is  that  there  are  two  types  of 
COPPERHEAD  missions  (priority  and  target  of  opportunity)  and  the  data  from  Ft.  Sill 
did  not  categorize  these  two  types.  Priority  COPPERHEAD  missions  are  preplanned 
missions  with  preassigned  targets.  The  mission  data  is  stored  until  the  target  appears; 
the  mission  is  then  reactivated  and  carried  to  its  conclusion.  A  target  of  opportunity 
mission  is  not  a  planned  mission  but  occurs  when  a  target  appears  at  an  opportune  time 
and  place.  Target  of  opportunity  missions  require  a  longer  processing  time  by  the  FIST 
than  priority  COPPERHEAD  missions  once  the  target  is  acquired. 

TABLE  5.  COPPERHEAD  MISSIONS 
(REVIEW  &  AUTOMATIC) 

(MEDIAN  SERVICE  TIME) 


MODE 


MISSION  WORKLOAD 

GROUP  1 

GROUP  2 

CPU 

7.0 

73.0 

FO  +  CPH 

7,0 

92.0 

FO  +  ARMOR  +CPH 

8.0 

45.0 

CPH 

7.0 

68.5 

FO  +  CPH 

8.0 

70.0 

FO  +  ARMOR  +  CPH 

0.0 

66.0 

MB 


VI.  CONCLUSIONS  Baaed  on  the  results  of  the  analysis  of  the  limited  database 
obtained  from  the  Field  Artillery  Board,  the  FIST  HQ  demonstrated  its  ability  to 
perform  fire  support  coordination.  The  FIST  HQ  ability  to  service  fire  missions  was  not 
affected  by  different  FIST  IIQ  configurations.  The  FIST  did  perform  efficient  fire 
support  coordinations  when  the  FIST  Chief  was  not  present  and  when  two  members  of 
the  FIST  were  not  available  because  the  G/VLLD  was  dismounted.  Although  Mission 
Workload  and  Mode  of  FO  Control  were  significant,  the  largest  median  service  time 
observed  was  only  €8.0  seconds.  This  occurred  when  the  FIST  HQ  had  to  input  the 
voice  messages  from  the  Armor. 

The  number  and  types  of  missions  processed  simultaneously  influenced  the  FIST 
IIQ  ability  to  service  FO  and  Armor  missions.  However,  Mission  Workload  did  not 
affect  the  two  types  of  COPPERHEAD  missions  that  were  categorized  using  cluster 
analysis.  Based  on  this  statistical  technique,  COPPERHEAD  missions  were  shown  to 
not  be  affected  by  the  FIST  DM1)  mode  of  control.  In  fact,  FIST  DMD  mode  of  control 
only  affected  the  mechanized  infantry  FO  missions. 

Finally,  the  automatic  reduction  system  proved  to  be  a  useful  tool  for  data 
collection  and  reduction  of  field  data  and  the  ability  to  perform  a  controlled  experiment 
during  a  Odd  test  was  demonstrated  with  overwhelming  success.  However,  it 
demonstrates  the  need  for  more  sophisticated  MOP’s  than  simply  speed  of  service. 
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INTRODUCTION 


1.  The  purpose  of  Chis  study  was  to  explore  methods  for  predicting  mid 
reducing  the  error  in  neasured  explosion  wave  data  due  to  the  presence  of  n 
wind  generated  random  aea  suface.  In  this  analysis,  the  model  to  be  used  for 
representing  the  mixture  of  random  and  deterministic  waves  is  the  linear 
superposition,  or  adding  together,  of  explosion  wave  and  random  wave  amplitude 
functions.  An  analysis  method  based  on  the  linear  superposition  model 
requires  the  assumption  that  frequency  components  of  interest  in  the  explosion 
generated  waves  are  contained  in  a  pertain  frequency  band.  Errors  resulting 
from  the  random  wave  frequencies  outside  the  specified  band  of  Interest  arc 
then  eliminated  by  means  of  band  pass  filters.  Resulting  estimates  for  the 
explosion  wave-form  will  be  biased  by  smoothing  inherent  in  the  filtering 
process.  This  bias  effect  is  reduced  by  observing  the  affect  that  the 
filtering  process  has  on  theoretical  models  for  sxploeion  waves.  Theoretical 
models  have  been  shown  to  correspond  well  with  meaeur;ad  data  (Le  Mehaute  1971) 
malting  them  useful  ae  a  muans  of  astlmatinjg  bias  due  to  the  filtering  process. 
Correction  factors  foir.  a  specific  filter  and  sst  of  axploiion  wave  parameters 
ars  computed  and  u.eed  to  remove  fllttr  bias  from  sstimatss  of  maximum 
txploslon  wave  amplitude. 

2.  Expected  error  due  to  random  background  noies  can  be  predicted 
prior  to  a  test  using  the  energy  spectrum  of  the  ees  surface.  Thie  makes  it 
possible  to  make  GO  or  NO  GO  decisions  prior  to  testing  based  on  the  roughness 
of  the  soa  surface.  The  GO/NO  GO  model  developed  here  is  based  on  the 
parameterlsed  sea  surface  spectrum  developed  by  Ochl  and  Hubble  (1976).  An 
Interactive  computer  program  for  predicting  expected  background  error  is 
developed  and  examples  of  its  usage  ars  presented  in  a  latar  section. 

Wave  mixture  model 

3.  Let  the  discrete  time  aeries  assoclsted  with  the  random  sea  surface 
be  denoted  by  xn  and  the  discrete  version  of  ths  explosion  wave  time  series 
by  pn  for  n  ■  0,  1,  ...,  N  -  1  with  a  time  Increment  of  At  between 
successive  numbers  of  the  two  time  series.  Linear  superposition  of  the  two 
time  seriee  produces 


^on 


Pn  +  xn  »  n  "  °*  l» 


,  N  -  1 


1.1 


360 


* 

,‘q  \%  ',*•  *e  /<*'•»  T/ *  ‘V**.  *"•  ^*4  m* 


It  is  assumed  that  each  variable  in  the  time  series  xn  is  distributed  as  u 

2 

Gaussian  random  variable  with  mean  zero  and  variance  o  .  Hie  time  series 
pon  represents  the  observed  mixture  of  noise  and  explosion  waves.  This 
variable  pQn  is  also  distributed  Geueslan  with  mean  pn  and  variance  o^. 
The  dlacrete  fourier  transform  (DPT)  for >  pon  is 


N  -  1 

■  At  l  p  s' 
om  n  -  0  on 


-12irmn/N 


1.2 


or 


pom  "  pm  +  ^m 


1.3 


Where 


P  -  At  J  p 
om  4  «  *^on 

n  ■  o 


1.4 


N  -  1 

X_  «•  At  I  x„e 
m  n  ■  0  n 


,-i2innn/N 


1.5 


are  the  DFTs  for  pn  and  xn  respectively.  Subscript  m  refers  to  frequency 
veluee  f  ■  mAf  for  Af  ■  l/(NAt). 


4.  Define  the  estimate  of  the  DPT  for  Pm  to  be 


Pm  -  Prtm  D  ,  ra 
m  om  m 


0,  l,  —  ,  N  -  1 


1.6 


for 


m 


-  1.0,  Mj  $  m  ^  M2 

1.0,  N-M2<m<N-M1 
0.0,  all  other  values  of  m 


1.7 


Values  fj  ■  Mj  Af  and  f2  ■  M2  Af  define  the  frequency  Interval  of  interest 
for  the  data  to  be  analysed.  Than,  tha  estimate  for  the  time  aeries  pn  Is 


the  Inverse  DFT  of  Pm  ,  given  by 


m 


•  D  e 
om  m 


i2nmn/N 


1.8 
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Note  that  since  p  Is  a  linear  combination  of  linear  combinations  of 

0  A 


Cuuaaian  random  variables  pon  ,  n  »  0 ,  1,  —  ,  N  -  1  then  p  is  also 


distributed  Gaussian.  Substituting  equation  1.3  into  1.8,  the  estimated  time 


series  can  be  rewritten  ast 


p  “  p  B  +  c 
*n  n 


where 


N  -  1 

8n  -  Lf  l  PM  (D  -  1.0) 
n  ■  lB  .  0  m  m 


i2nmn/N 


N  -  1 

cn  ■  Af  l  X  D  e 
n  m  -  0  m  m 


12irmn/N 


3.  The  term  is  referred  to  at  the  estimate  bias  term  and  e  In 
O'  .  1  n 


called  the  random  term.  Statistical  expectation  of  pn  is 


S[p„]  ■?„  +  #„ 


since  E[e  ]  “0,0  end  eince  the  other  terms  ere  deterministic.  Variability 
n  A 


of  tha  sstlmate  pn  is  glvsn  by  its  variance,  which  le 


N  -  1 


Vet  [pi  -  Li  l  Sm  D 
n  m  -  0  m  n 


where  8  le  the  discrete  version  of  the  power  spectral  density  of  the  random 


see  surface.  Standard  deviation  of  p_  Is 

n 


s  »J7m 

n  \  n 


6.  Assuming  that  oR  and  SR  art  known,  than  the  (l-a/2)  x  100 


percent  confidence  interval  for  pn  le 


<pn’  *n~#n  Z1  -  a/2  •  pn  ‘  *n  +  an  Z1  -  a/2> 


$ 


smesmsm^ 


wyfJRyj  w-.. 


where  Zj  _  is  the  1  -  a/2  percentage  point  for  a  standard  normal 

distribution. 

7.  In  an  applied  setting,  o  la  not  a  known  quantity;  however,  it 

”  A 

can  be  estimated  by  replacing  Sm  with  SR  ,  the  estimated  energy  spectral 
density  in  equation  1.13. 

8.  The  bias  term  0n  will  be  estimated  by  observing  the  affect  that 
filtering  has  on  functions  that  are  used  to  model  explosion  wove  time  sert«s 
for  given  explosive  yields,  water  depths,  and  measurement  location. 

9.  To  simplify  the  discussion  of  the  random  error  in  later  parts  of 
this  report,  the  bias  term  will  be  included  in  estimates  of  the  explosion 
wave  time  series  and  the  unbiased  estimate  of  p^  will  then  be 

A  A 

p  ■  p  -  S  1,16 

*un  *n  1,40 

with  confidence  interval 

pun  *  °n  Z1  -  a/2  1,17 

Explosion  wave  model 

10.  There  are  several  functions  that  are  used  to  model  explosion  wuvo 
amplitude.  The  function  used  here  for  testing  and  bias  estimation  is  the  most 
suitable  of  the  available  models  (La  Mehauta  1971).  Explosion  wave  amplitude 
as  a  function  of  time  and  distance  from  the  explosion  is  written  (dimension¬ 
less  form) 

n(r,t)  ■  K  cos  (Kr  -  t(tanh  K))1^2  1.18 


where 


and 


*  ■  H  J3  <«> 


1.19 


R  *  radius  of  initial  cavitation 
n  ■  amplitude  of  initial  cavitation 
K  ■  wave  number 


r  -  distance  at  which  amplitude  is  derived 


V  (K)  -  wave  group  velocity 
J3  (KR)  ■  Bessel  function  of  the  first  kind  of  order  3 

11*  the  parameters  Hated  above  are  dimensionless  resulting  in  a 
dimensionless  value  for  aquation  1.18.  Zn  order  to  obtain  the  dimension/'] 
version,  equation  1.18  is  multiplied  by  the  water  depth  at  which  the  explosion 
occurs.  Equation  1.18  is  evaluated  numerically  for  given  values  of  the 
parameters  to  provide  test  data  for  the  analysis  techniques  of  this  study. 
Random  noise  simulation 

12.  Frequency  domain  simulation  of  Gaussian  random  time  series  makes 
use  of  the  statistical  identities  between  linear  combinations  of  Gaussian 
variables.  That  is,  any  linear  combination  of  Independent  Gaussian  random 
variables  is  also  Gaussian  with  mean  and  variance  that  are  linear  combinations 
of  the  means  and  varlSnces  of  the  original  random  variables.  Discrete  Fourier 
Transform  (DFT)  pairs  are  linear  combinations  and,  therefore,  have  Identities 
described  above.  A  Gaussian  time  aeries  with  a  given  theoretical  power 
spectrum  can  be  shown  to  have  a  DFT  that  is  Gaussian  with  a  variance  that  is 
rslated  to  ths  power  epectrum  of  the  time  aeries.  That  la,  if  X_., 

9 

n  -  0,  1 . N  -  1  is  Gaussian  with  mean  aero  and  variance  o  and  results 

from  d  random  process  with  power  spectral  danaity  8(f),  then  the  DFT  of  Xn 
given  by  Um  -  i  Vm  has  real  and  imaginary  parts  Um  ,  Vm  that  are  also  mean 
zero  Gaussian,  with  varlancat 

'  NA  t  S  m  -  0,  N/2 

Vsr(U  )  -  21 

m  NAt  S  0  <  m  <  N/2 

m 

.  2 

'0  m  -  0,  N/2 

Var(V  )  ■  22 

m  NAt  8  0  <  m  <  N/2 

n 

.  2 

for  Sffl  the  discrete  version  of  ths  powsr  spectrum  S( f )  .  It  is  also  known 
that  (Um  ,  Vffl)  and  (Um»  ,  Vm/)  are  statietically  independent  for  m**m'  0  < 
m,  m'  <  N/2,  end  Uffl  is  independent  of  Vm»  for  all  m,  m'  .  The  values  for 
m  >  N/2  can  be  found  by  the  identity  um  "  uN-m  and  vm  "  vN-m  * 


13.  The  steps  for  simulating  a  Gaussian  random  time  series  with  power 
spectral  density  S(f)  ares 

(1)  Generate  two  statistically  Independent  sequencee  of 
independent  mean  taro  Gaussian  random  numbers,  say,  Xl^  and  XIjj  for 
0  <  m  <  N/2 


(2)  Ut  Un  -  XR,,  (Sit  -$> 


'  XIm  <“*  -?> 


0  <  m  <  N/2 


Um  "  X\  8m> 


a  ■  0,  N/2 


(3)  Apply  DFT  to 

'  \ " ».  - 1 

to  obtain  the  resulting  times  aeries  XJV  ,  n  ■  0,  1,  ...  ,N-lof  equation 
1.5.  Finally,  add  the  time  series  from  equation  1.18  to  to  obtain  the 
desired  mixture  of  a  Gaussian  random  sea  surface  with  explosion  generated 
water  waves. 

GO/NO  GO  analysis 

14.  Equation  1.13  gives  the  expected  error  due  to  the  presence  of 
random  background  noise  in  the  measured  explosion  wave  time  history.  The 
associated  confidence  Interval  of  equation  1.15  represents  the  interval  that 
will  capture  the  true  explosion  wave  time  history,  Pn  ,  (1  -  a/2)  x  100 
percent  of  the  time.  If  the  power  spectral  density  of  the  random  sea  surface 
is  known  or  specified  by  one  of  the  commonly  used  parametric  forms,  than 
equations  1,13  through  1.17  can  be  computed  to  provide  an  estimate  of  the 
statistical  error  to  be  expected  in  filtered  explosion  wave. 

15.  A  parameterized  spectrum  known  as  the  Ochl  Hubble  (1976)  six 
parameter  spectrum  was  chosen  for  use  in  this  analysis.  This  six  parameter 
spectrum  includes  a  spectral  peakednaes  parameter  that  determines  the  narrow 


bandedneas  of  the  spectrum.  A  peakedness  parameter  of  1.0  is  used  here 
resulting  in  a  spectrum  that  is  equivalent  to  Bre..schnieder  and  Pierson 
Mottowitt  type  spectral  densities.  Parameters  needed  for  input  to  the 
parameterized  spectrum  are  the  mean  period  for  long  period  waves  generated  by 
distant  storms  (swell)  given  by  Ts  ,  mean  period  for  locally  generated 
waves  ,?w,  significant  wave  height  for  swell  Hs  ,  and  significant  wave  height 
for  sea  Ky  ,  Figuru  1  is  &  plot  of  the  quantity  on  ^  from  equation 
1,17  for  varying  vaiuns  of  spectral  parameters.  Examples  and  instructions  for 
t hf  GO/NO  GO  device  program  are  listed  in  Appendix  A. 
gynthmtlc  data  analysis 

?  i  ■ 

16.  simulations  were  computed  for  varying  measurement  distances  from 
the  explosion  source.  Distance  values  were  500,  800,  1,000,  and  2,000  ft. 
There  were  five  simulation*  for  eech  location,  each  representing  e  different 
random  background  wave  sequence.  Random  background  waves  were  generated 
according  to  the  method  of  section  4  using  an  Ochi  Hubble  type  bimodal 
spectrum  with  significant  height . for  swell  equal  to  1.0  ft,  significant  height 
for  local  seas  equal  to  .75  ft,  mean  period  for  swell  at  12.5  sec,  and  mean 
period  for  sees  at  5  esc. 

17.  Two  methods  were  used  for  estimating  explosion  wave  time 
histories.  The  first  being  the  boxcar  type  filter  described  in  section  2;  the 
second  method  was  an  8  pole  Butterworth  digital  filter  with  maximum  flatness 
in  the  passband  and  stop  band. 

18.  Plots  of  theoretical,  observed,  end  estimated  time  series  along 

with  random  noise  xn  from  equation  1.1,  random  error  cn  from  equation 

1.11,  and  bias  0n  from  equation!. 1C  are  presented  in  Appendix  B.  Of 

particular  interest  is  the  maximum  wave  amplitude  for  each  simulation.  Values 

for  theoretical,  observed  and  estimated  maximum  wave  amplitudes  are  given  in 

Table  1  and  Table  II .  The  tables  are  also  Include  the  associated  values  for 
2  2 

o  ,  sea  surface  variance  or  mean  square  error,  and  o  ,  error  variance  or 

b 

mean  square  of  the  random  error  term  e  from  equation  1.11,  The  theoretical 
2  n 

value  for  o  of  equation  1,13  from  the  GO/NO  GO  example  of  Appendix  A  is 
2  n  2  2 

o  ■  0.039  .  Since  o  is  the  sample  variance  and  a  is  the  theoretical 
n  e  n 

variance  of  the  explosion  wave  estimate  time  series,  their  values  should  be 

2 

close.  Note  that  values  for  o  generally  scatter  around  0.039.  Average 
2  2  c 

values  for  a  and  o  over  eech  set  of  twenty  simulation  demonstrate  that 

fc 


1.0  ft.  The  Significant  l 


TABLE  1 
Boxcar  Filter 


R 

Pmax 

■m 

B 

obsmax 

- 7* 

c 

500 

9.07 

8.87 

8.74 

.096 

.040 

500 

9.07 

9.06 

8.85 

.100 

.042 

500 

9.07 

9.60 

9.72 

.044 

500 

9.07 

8.99 

8.96 

.102 

.053 

500 

5.07 

8.74 

8.43 

.092 

.042 

800 

5.59 

5.37 

5.58* 

.095 

.039 

800 

5.59 

5.55 

5.74 

.102 

.043 

800 

5.59 

5.50 

5.36 

.084 

.043 

800 

5.59 

5.66 

5.32 

.097 

.040 

800 

5.59 

5.29' 

5i.53*  , 

.088 

.040 

1000 

4.15 

4.25 

4.27 

.094  ' 

.039 

LQ00 

4.15 

4.55 

4.67 

.103 

‘.031 

1000 

4.15 

3.96 

4.12* 

.103 

.044 

1000 

4.15 

3.87 

3.88* 

.095 

.044 

1000 

4.15 

3.85 

4.25* 

.096 

.040 

2000 

2.25 

2.33 

2.18* 

.102 

.046 

2000 

2.25 

2.07 

1.68 

.106 

.048 

2000 

2.25 

2.47 

1.96 

.091 

.039 

2000 

2.25 

2.26 

2.38 

.103 

.038 

2000 

2.25 

2.29 

2.37 

.109 

.050 

F  ■  Theoretical  aaxiaua  exploeion  wave  eaplitude. 
.aax 

■  Estimated  aaxiaua  explosion  wave  eaplitude. 
*obsaax  "  Obeervad  aaxiaua  explosion  wave  eaplitude. 
*  Esc lasted  wave  eaplitude  le  worse  than  observed. 


max 


max 


Thaoratical  maximum  axplosion  vava  amplltuda. 
Estimatad  maximum  axploalon  vava  amplltuda. 


Pobamax  "  Obaarvad  maximum  axplosion  vava  amplltuda. 
*  Eatlmatad  vava  amplltuda  is  vorsa  than  obsarvad. 
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Che  Butterwcrth  filter  does  better  than  the  Boxcar  in  reducing  the  overall 
estimate  error. 

(1)  Boxcar 

2 

Average  (o  )  ■  0.098 

2 

Average  a£  •  0.042 

(2)  Butterworth 

Average  (o2)  -  0.096 

2 

Average  o6  -  0.029 

19.  Boxcar  filter  estimates  are  closer  to  the  theoretical  value  of  elm 
maximum  wave  amplitude  than  the  observed  maximum  about  70  percent  of  the 
time.  The  Butterworth  does  better  than  no  filter  90  percent  of  the  time. 
Results  and  conclusions 

20.  Random  background  noise  effects  on  measured  explosion  wave  time 
series  can  be  reduced  by  means  of  band  pass  filter  techniques.  Ihe 
Butterworth  type  filter  tends  to  do  a  better  job  at  reducing  error  tlum  a 
Boxcar  filter  in  frequency  space. 

21.  Ihe  estimate  error  for  the  filtered  explosion  wave  estimates  can  ho 
predicted  using  the  power  spectral  density  of  the  background  sea  surface.  In 
this  way,  GO  or  NO/GO  decisions  can  be  made  during  testing  based  on  sea 
surface  conditions  at  the  time. 

22.  A  GO  or  NO/GO  computer  program  is  given  along  with  self  explanatory 
documentation  In  Appendix  A.  Ihe  program  is  designed  to  provide  quick  easy 
error  predictions  using  any  computer  that  usee  FORTRAN. 
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APPENDIX  A 


Appendix  A  consist*  of  the  listings  for  the  QO/NO  00  computer  program  and 
example  of  its  usage. 


GO  MO  GO  COMPUTER  PROGRAM 

The  GO  NO  GO  program  was  developed  on  the  Honeywell  DPS-1  system  at  the 
Waterways  Experiment  Station.  The  program  Is  written  In  standard  FORTRAN 
that  is  easily  adapted  to  any  system  that  supports  standard  FORTRAN  IV. 

There  may  be  some  minor  changes  in  the  Input  and  output  conventions  and  line 
numbering.  For  DEC  VAX  systems  the  PRINT  commands  should  be  replaced  by 
PRINT  *  commands  and  the  line  numbers  removed. 


IOC  PROGRAM  GO  NO  GO 

20  F1-.08 

30  F2-.4 

40  PRINT, "THIS  PROGRAM  COMPUTES  THE  EXPECTED  ERROR  FOR  A" 

50  PRINT, "DETERMINISTIC  WAVEFORM  THAT  IS  CONTAMINATED  WITH" 

60  PRINT, "GAUSSIAN  RANDOM  NOISE" 

70  PRINT, "THE  INPUT  PARAMETERS  ARE  THE  FOUR  PARAMETERS" 

80  PRINT, "ASSOCIATED  WITH  THE  WAVE  SPECTRUM  FOR  A  MIXTURE" 

90  PRINT, "OF  LONG  PERIOD  SWELL  AND  LOCALLY  GENERATED  SEAS" 

100  PRINT, "THE  INPUT  UNITS  DETERMINE  THE  UNITS  OF  THE  OUTPUT" 

110  PRINT, "THE  PARAMETERS  ARE l " 

120  PRINT, "TS  -  MEAN  PERIOD  FOR  .SWELL" 

130  PRINT, "TW  -  MEAN  PERIOD  FOR  SEAS" 

140  PRINT, "HS  -  SIGNIFICANT  WAVE  HEIGHT  FOR  SWELL" 

150  PRINT, "HW  -  SIGNIFICANT  WAVE  HEIGHT  FOR  SEA" 

160  PRINT, "IF  YOU  WISH  TO  RUN  THIS  PROGRAM  TYPE  1  " 

170  PRINT, "AT  THE  PROMPT  FOLLOWED  BY  A  CARRIAGE  RETURN" 

180  PRINT, "OTHERWISE  TYPE  0  FOLLOWED  BY  A  CARRIAGE  RETURN" 

190  READ, I RUN 

200  IF( I RUN  .NE .  1 ) STOP 

210  5  PRINT, "TYPE  THE  WAVE  SPECtRUM  PARAMETERS  ONE  TO  A  LINE" 

220  PRINT, "ACCORDING  TO  THE  PROMPT  MESSAGES" 

230  6  PRINT , "TYPE  THE  MEAN  PERIOD  FOR  SWELL" 

240  READ,TS 

250  IF(TS.GE.5.0)GO  TO  7 

260  PRINT, "ILLEGAL  VALUE  FOR  TS,  RETYPE" 

270  GO  TO  6 

280  7  PRINT, "TYPE  THE  MEAN  PERIOD  FOR  SEAS" 

290  READ,TW 

300  IF(TW.GT.0 .0 .AND. TW.LT. 10.0)  GO  TO  8 

310  PRINT, "ILLEGAL  VALUE  FOR  TW,  RETYPE" 

320  GO  TO  7 

330  8  PRINT, "TYPE  THE  SIGNIFICANT  HEIGHT  FOR  SWELL" 

340  READ, HS 

350  IF(HS.GT.O.O)  GO  TO  9 

360  PRINT, "ILLEGAL  VALUE  FOR  HS,  RETYPE" 

370  GO  TO  8 

380  9  PRINT, "TYPE  THE  SIGNIFICANT  HEIGHT  FOR  SEA" 

390  READ , HW 

400  IF( HW .GT .0 .0 )GO  TO  10 

410  PRINT, "ILLEGAL  VALUE  FOR  HW,  RETYPE" 

420  GO  TO  9 

430  10  PRINT, "THE  DEFAULT  VALUES  FOR  THE  FREQUENCY  BAND  OF" 
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440  PRINT, "THE  DETERMINISTIC  WAVEFORM  ARE  »" 

450  PRINT, "FI  -  " , F 1 , "  F2  -  ",F2 

460  PRINT, "IF  YOU  WISH  TO  RESET  THE  VALUES  FOR  FI  AND  F2" 

470  PRINT, "TYPE  1  FOLLOWED  BY  A  CARRIAGE  RETURN  OTHERWISE" 

480  PRINT, "TYPE  0  FOLLOWED  BY  A  CARRIAGE  RETURN" 

490  READ, ICHNG 

500  IF( ICHNG .NE. 1 )GO  TO  15 

510  PRINT, "TYPE  THE  NEW  VALUE  FOR  FI" 

520  READ, FI 

530  PRINT, "TYPE  THE  NEW  VALUE  FOR  F2" 

540  R£AD,F2 

550  15  CONTINUE  ,  . 

560  PRINT, "PROGRAM  IS  RUNNING  PLEASE  WAIT" 

570  PRINT,".-.-.-.-.-.-.-.-.-.-.-.- 

57 1C  COMPUTE  THE  ERROR  THEN  ADJUST  BY  THE  STANDARD 
572C  NORMAL  PERCENTAGE  POINT  TO  OBTAIN  A  90  PERCENT 

573C  CONFIDENCE  LIMIT. 

580  CALL  ERRP(F1 ,F2,TS,TW,HS,HW,E) 

590  ERROR*SQRT( E) 

600  CON-1. 645*ERROR 

610  ■  PRINT, "THE  EXPECTED  STANDARD  DEVIATION  FOR  THE" 

620  PRINT, "ESTIMATE  OF  THE  DETERMINISTIC  WAVEFORM  IS  " 

6  30  PRINT, "APPROXIMATELY  S  -  ",  ERROR 

640  PRINT, "a*********************************"  •  . 

650  PRINT, "THE  90  PERCENT  CONFIDENCE  INTERVAL  FOR  THE  ESTIMATE" 

660  PRINT, "IS  PLUS  OR  MINUS  THE  VALUE  C  -  ",CON 

670  PRINT,"***********************************************" 

680  PRINT, "IF  YOU  WISH  TO  OBTAIN  ANOTHER  SET  OF  VALUES  TYPE" 

690  PRINT, "1  FOLLOWED  BY  A  CARRIAGE  RETURN  OTHERWISE  TYPE  0" 

700  PRINT, "THEN  A  CARRIAGE  RETURN" 

710  READ, I  RUN 

720  IF(IRUN.NE.l)  STOP 

730  GO  TO  5 


750  SUBROUTINE  ERRP( FI ,F2 ,TS,TW,HS,HW, E) 

75ic******************«**********t ***********************  ******* 

75 1C  THIS  SUBROUTINE  COMPUTES  THE  AREA  UNDER  AN  OCHI- 

752C  HUBBLE  TYPE  SPECTRUM  BETWEEN  THE  FREQUENCY  VALUES 

753C  FI  AND  F2  FOR  INPUT  PARAMETERS, 

754C 

755C 

755C  TS  -  MEAN  PERIOD  FOR  SWELL 

755C  TW  -  MEAN  PERIOD  FOR  LOCAL  SEAS 

7 55C  HS  -  SIGNIFICANT  WAVE  HEIGHT  FOR  SWELL 

755C  HW  *  SIGNIFICANT  WAVE  HEIGHT  FOR  LOCAL  SEAS 

755C 

755C  *  * 

755C ************************************************************ 
760  DELF"( F2-F1 )/I00  • 

770  SUM-0.0 

780  DO  10  1-1,100 

790  F-Fl+( I-1)*DELF 

800  ,  SUM-SUM+SPEC(TS,HS,F)+SPEC(TW,HW,F) 

810  10  CONTINUE 

820  E-SUM*DELF 

830  RETURN 

840  END 

850  FUNCTION  SPEC(T,H,F) 

8510*****************************************.******** 

860C  COMPUTES  THE  OCHI-HUBBLE  TYPE  SPECTRUM  FOR 

86 1C  PARAMETERS, 

862C 

862C  T  -  MEAN  PERIOD 

862C  H  -  SIGNIFICANT  WAVE  HEIGHT 

862C 

86 2C* ****************************************  ******** 

860  A-(5./(4 .*T**4> ) 

870  B-A* ( (H/4 .)**2)/{F**5 .) 

880  ARG— A/(F**4.) 

890  IF(ARG.GE,-50 .0)  GO  TO  5 

900  SPEC-0.0 

910  RETURN 

920  5  SPEC-4 .0*B*EXP(ARG) 

930  RETURN 

940  END 
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GO/ NO  GO  EXAMPLE 


The  GO/NO  GO  example  begins  with  an  FRN  command.  This  command  Is  specific  to 
tha  Honeywell  DPS-1  system  and  maana  gat  the  program  GO  NO,  compile  It,  nnd 
run  It.  The  rest  of  the  example  Is  typical  Interactive  session  with  the  GO/ No 
GO  package • 


-FRN  GONO 

THIS  PROGRAM  COMPUTES  THE  EXPECTED  ERROR  FOR  A 
DETERMINISTIC  WAVEFORM  THAT  IS  CONTAMINATED  WITH 
GAUSSIAN  RANDOM  NOISE 

THE  INPUT  PARAMETERS  ARE  THE  FOUR  PARAMETERS 
ASSOCIATED  WITH  THE  WAVE  SPECTRUM  FOR  A  MIXTURE 
OF  LONG  PERIOD  SWELL  AND  LOCALLY  GENERATED  SEAS 
THE  INPUT  UNITS  DETERMINE  THE  UNITS  OF  THE  OUTPUT 
THE  PARAMETERS  ARE  t 
TS  -  MEAN  PERIOD  FOR  SWELL 
TW'  -  MEAN  PERIOD  FOR  SEAS 
HS  -  SIGNIFICANT  WAVE  HEIGHT  FOR  SWELL 
HW  -  SIGNIFICANT, WAVE  HEIGHT  FOR  SEA 
IF  YOU  WISH  TO  RUN  THIS  PROGRAM  TYPE  I 
AT  THE  PROMPT  FOLLOWED  BY  A  CARRIAGE  RETURN 
OTHERWISE  TYPE  0  FOLLOWED  BY  A  CARRIAGE  RETURN 
■1 

TYPE  THE  WAVE  SPECTRUM  PARAMETERS  ONE  TO  A  LINA' 
ACCORDING  TO  THE  PROMPT  MESSAGES 
TYPE  THE  MEAN  PERIOD  FOR  SWELL 
■12.5 

TYPE  THE  MEAN  PERIOD  FOR  SEAS 
■5  .0 

TYPE  THE  SIGNIFICANT  HEIGHT  FOR  SWELL 

■1.0 

TYPE  THE  SIGNIFICANT  HEIGHT  FOR  SEA 
-.75 

THE  DEFAULT  VALUES  FOR  THE  FREQUENCY  BAND  OF 
THE  DETERMINISTIC  WAVEFORM  ARE  » 

FI  ■  0.80000000E-01  F2  ■  0.40000000E  00 

IF  YOU  WISH  TO  RESET  THE  VALUES  FOR  FI  AND  F2 
TYPE  1  FOLLOWED  BY  A  CARRIAGE  RETURN  OTHERWISE 
TYPE  0  FOLLOWED  BY  A  CARRIAGE  RETURN 
■1 

TYPE  THE  NEW  VALUE  FOR  FI 

■.11 

TYPE  THE  NEW  VALUE  FOR  F2 
■  .25 


PROGRAM  IS  RUNNING  PLEASE  WAIT 


THE  EXPECTED  STANDARD  DEVIATION  FOR  THE 
ESTIMATE  OF  THE  DETERMINISTIC  WAVEFORM  IS 
APPROXIMATELY  S  ■  0 . 1973928 3E  00 

********************************** 

THE  90  PERCENT  CONFIDENCE  INTERVAL  FOR  THE  ESTIMATE 
IS  PLUS  OR  MINUS  THE  VALUE  C  ■  0.32471121E  00 

A***************** ***************************** 

IF  YOU  WISH  TO  OBTAIN  ANOTHER'  SET  OF  VALUES  TYPE 
1  FOLLOWED  BY  A  CARRIAGE  RETURN  OTHERWISE  TYPE  0 
THEN  A  CARRIAGE  RETURN 

■1 

TYPE  THE  WAVE  SPECTRUM  PARAMETERS  ONE  TO  A  LINE 
ACCORDING  TO  THE  PROMPT  MESSAGES 
TYPE  THE  MEAN  PERIOD  FOR  SWELL 
-10.0 

TYPE  THE  MEAN  PERIOD  FOR  SEAS 

■6  .0 

TYPE  THE  SIGNIFICANT  HEIGHT  FOR  SWELL 

■1 . 

TYPE  THE  SIGNIFICANT  HEIGHT  FOR  SEA 
-.5 

THE  DEFAULT  VALUES  FOR  THE  FREQUENCY  BAND  OF 

THE  DETERMINISTIC  WAVEFORM  ARE  t 

FI  -  0 . 11000000E  00  F2  -  0.25000000B  00 

IF  YOU  WISH  TO  RESET  THE  VALUES  FOR  FI  AND  F2 

TYPE  1  FOLLOWED  BY  A  CARRIAGE  RETURN  OTHERWISE 

TYPE  0  FOLLOWED  BY  A  CARRIAGE  RETURN 

*0 

PROGRAM  IS  RUNNING  PLEASE  WAIT 


THE  EXPECTED  STANDARD  DEVIATION  FOR  THE 
ESTIMATE  OF  THE  DETERMINISTIC  WAVEFORM  IS 
APPROXIMATELY  S  ■  0.21594351E  00 

THE  90  PERCENT  CONFIDENCE  INTERVAL  FOR  THE  ESTIMATE 

IS  PLUS  OR  MINUS  THE  VALUE  C  -  0.35522708E  00 

*********************************************** 


APPENDIX  B 


Tim*  Serial  Plot*  for  Synthetic  Wave  Data  Analytic 

Time  eerlee  plots  Include  examples  of  the  theoretical  explosion  waveform 
versus  the  explosion  waveform  mixed  with  random  noise,  random  noise,  filtered 
estimate  versus  theoretical  waveform,  random  error  or  random  noise  after 
filtering,  and  bias  due  to  filtering.  The  plots  represent  a  time  series  that 
would  result  if  a  device  began  measuring  the  background  noise  exactly  SO  sec 
before  the  first  explosion  wave  reaches  the  measurement  location. 
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WRVEF-ORM  COMPARISON 


Figure  B9.  Tiaie  series  plot  for  Boxcar  Estimate  at 
800  Ft  f com  the  explosion  source 
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Figure  BIO.  Random  error  for  the  estimated  time  series  in  Figure  B9 
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WAVEFORM  COMPARISON.  THEOR 
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or  the  Butte  worth  Estimate 
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Figure  B26.  Random  error  from  the  estimated  time  series  of  Figure  B25 
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WAVEFORM  i:0!ir,RR  I  SOM  .  THEORY  VS  t  ST  I  MATE 
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Figure  B31.  Tine  series  plot  for  the  Sutterworth  Estimate 
at  2000  ft  from  the  explosion  source 


WRVEf-  ORM  COhPRRISON.  THEORY  VS  FSTIMRTE 


A  VARIABLE  SELECTION  MODEL  BUILDING 
TECHNIQUE  FOR  RADAR  MEASUREMENT  BIAS  ESTIMATION 


WILLIAM  S.  AGEE 
ANDREW  C.  ELLINGSON,  JR. 
ROBERT  H.  TURNER 


National  Range  Operations  Directorate 
White  Sands  Missile  Range,  New  Mexico 


ABSTRACT.  Measurements  of  range,  azimuth,  and  elevation  from  several 
dlfferenFraHars  are  combined  to  estimate  the  cartesian  coordinates  of  a 
vehicle  trajectory.  Since  the  sequence  of  times  t^ ,  1*1,  2,-—  ,N  which 

cover  the  entire  trajectory.  Since  the  measurements  are  subject  to 
systematic  errors  as  well  as  random  measurement  errors,  the  systematic 
error  parameters  (biases)  are  also  estimated,  The  resultlnq  estimation 
problem  Is  a  combined  linear  and  nonlinear  problem  In  which  the  trajectory 
coordinates  appear  nonllnearly  In  the  measurements  and  the  biases  appear  as 
linear  parameters  In  the  measurements.  Applications  of  the  above  estimation 
very  often  result  In  Ill-conditioned  linear  equations  for  estimating  the 
radar  biases,  producing  erroneous  bias  estimates.  The  problem  of  111- 
conditioning  caused  by  mul tl coll Ineorlty  among  the  terms  Included  In  the 
bias  model,  Is  treated  by  using  a  backward  elimination  method  for  the 
selection  of  Independent  variables  to  be  Included  In  the  radar  measurement 
bias  model.  The  method  Is  Illustrated  with  examples  from  WSMR  radar  tracking 
missions. 

KEYJJORDS.  Mul 1 1  col  1 Inearlty,  trajectory,  estimation,  regression, 
variable  selection,  stepwise  regression 
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/>  VARIABLE  SELECTION  MODEL  BUILDING  1ECHNIQUE  FOR  RADffi  MEASUREMENT  BIAS  ESTIMATION 


INTRODUCTION. 

The  WSMR  radar  Best  Estimate  of  Trajectory  (BET)  program  combines 
the  range,  azimuth,  and  elevation  measurements  from  all  radars  tracking 
an  object  to  optimally  estimate  a  set  of  smoothed  cartesian  positions, 
velocities,  and  accelerations  of  the  object  at  each  measurement  time  tj. 
The  radar  measurements  are  subject  to  systematic  error  as  well  as  a 
random  error  component.  The  radar  systematic  errors  are  usually 
referred  to  as  measurement  biases.  When  an. object  Is  being  tracked  by 

l 

multiple  radars  and  the  relative  geometry  between  the  object  trajectory 
and  the  radars  Is  sufficiently  good,  a  measurement  bias  for  each  range, 
azimuth,  and  elevation  measurement  can  be  estimated.  It  Is  Important, 
sometimes  absolutely  necessary,  for  the  success  of  the  multiple  radar 
BET  reduction  that  estimates  of  the  radar  measurement  bias  be  obtained. 
When  these  measurement  biases  are  estimated  by  least  squares,  the 
resulting  bias  estimates  are  often  erroneous  due  to  numerical  111- 
conditioning  of  the  least  squares  estimation  problem  and 
multlcoll Inearlty  between  the  terms  Included  In  the  bias  model  caused 
by  overfitting. 

We  have  attempted  to  treat  this  problem  of  erroneous  bias 
estimates  by  application  of  ridge  regression  techniques  [1]  and  by 
application  of  the  method  of  principal  components  [2].  We  have  had 
partial  success  with  each  of  these  methods  but  neither  has  proved 
satisfactory  for  automatic  trajectory  data  reduction.  Techniques  are 


developed  In  this  report  which  successfully  treat  the  problems  of 
Ill-conditioning  and  overfitting  In  the  radar  measurement  bias 
estimation  problem.  These  methods  are  being  Incorporated  Into  the 
WSMR  radar  BET.  These  techniques  are  based  on  the  use  of  modern , 
reliable  numerical  linear  algebra  algorithms  and  software  and  on  the 
development  of  a  statistical  model  building  technique  which  estimates 
only  the  radar  bias  terms  which  make  a  significant  reduction  In  the 
error  sum  of  squares. 

The  difficulties  with  the  radar  bias  estimates  are  Illustrated 
with  some  actual  WSMR  radar  data  sets.  These  data  sets  are  treated 
by  conventional  least  squares  estimation  methods  and  by  the  model 
building  technique  developed  herein. 

RADAR  MEASUREMENT  EQUATIONS 


Let  x,  y,  z  be  the  coordinates  of  an  object  In  a  local  cartesian 
coordinate  system  at  the  radar  site.  The  Ideal  mathematical  model  of 
the  radar  measurements  In  terms  of  these  cartesian  coordinates  Is: 


r(x)  *  (x2  +  ya  +  z2)* 


a (x)  ■  tan" 


e(x)  ■  tan' 


1  x 


(xa+  y2)! 


-  range 


-  azimuth 


-  elevation 


where  x  Is  the  position  vector  with  components  x,  y,  z.  The  radar  does  not 
measure  these  Ideal  values  of  range,  azimuth,  and  elevation,  but  Is  subject 
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to  a  measurement  error  which  we  assume  to  be  additive  and  Is  composed 
of  a  systematic  component  and  a  random  component.  Let  bR,  b^,  b^  be  the 
systematic  or  bias  components  of  error  for  range,  azimuth,  and  elevation, 
respectively.  Also,  let  eR,  e^,  eE  be  the  random  components  of  the 
measurement  error,  eR,  e^,  e^  are  assumed  to  be  Independent,  zero 
mean,  and  variance  c^,  o|,  respectively.  Then  the  measured 
values  of  range,  azimuth,  arid  elevation  are  modelled  as: 

R  ■  r(x)  +  bR  +  eR 

A  -  a(x)  +  bA  +  eA 

E  ■  e(x)  +  bt  +  eE 

LEAST  SQUARE  ESTIMATION  OF  OBJECT  POSITION  AND  RADAR  BIASES 

If  measurements  are  available  from  only  one  radar  there  Is  no 
potential  for  estimating  the  radar  measurement  biases,  bRI  bA,  b£  from 
radar  measurements  alone.  Thus,  estimates  of  the  object  cartesian 
position  obtained  from  the  single  radars  measurements  will  be  biased  If 
bR,  b^,  bjj  are  not  zero. 

Suppose,  however,  that  we  have  measurements  from  several  radars, 
say  M.  Let  the  radars  be  Indexed  by  a,  The  radar  measurement 

model  Is  now  written  as 

(1) 

a-l,M  (2) 

(3) 


Ra  ■  V*'  +  bRa  +  \ 


Aa  -  ac,<*>  +  bA  +  eA 
cx  a 

E0  ■  ea(x)  +  b£  t  «E 
ot  a 
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where  x  Is  no  longer  the  position  vector  of  the  object  In  a  local  radar 
cartesian  system  but  has  cartesian  components  in  a  common  reference 
coordinate  system.  The  functions  ra(x),  aa(x),  ea(x)  now  Involve  the 
position  of  the  radar  site  with  respect  to  the  origin  of  the  reference 
coordinate  system. 


r«<x>  ■  V'X4  +  y4  +  '4 


a«(x)  -  tan 


-1 


ea(x)  *  tan 


-1  *i 


'x4  +  *4 


whore  [xT/o  yT/(>  zT /o]  ■  [(x-xa)  (y-y0>  (i-za)]MwAi 

1^,  Is  a  rotation  matrix  from  the  local  radar  cartesian  system  to  the 

common  reference  system. 


Let  ha(x)  denote  the  vector  with  components  ra(x)»  aa(x)»  ea(x), 
b  the  vector  with  components  bR  ,  bA  ,  bE  ,  e  the  vector  with 

components  e„  ,  eA  ,  e,  and  m„  the  vector  with  components, 

Kot  Aa  ta  a 

Ra,  Aa,  Ea.  Then  the  radar  measurement  model  Is: 

\  +  ba  +  ®a’  a“1,M  W 

Suppose  we  have  radar  measurements  at  a  sequence  of  times  tj»  1*1 ,N. 
We  Index  the  position  vectors  and  measurements  with  t^  so  that  the 
radar  measurement  model  IsmJtJ  ■  h(x(tj)  +  b  +  e  (t,),  1-1, N. 


For  processing  the  radar  measurements  at  time  t^  further  package  the 
measurements  by  Introducing  the  vectors 


mT(t1)  •  (m^tj)  m2(ti )  —  m^t^)), 

hT(x(ti))  -  (h1(x(t1))  h2(x(t1 ))  —  hM(x(ti))), 

b^*  ■  (bj  bg  **■*  b^) i  and  e(t^)  ■  (e^(t^)  eg(t^)  mmm  e^(t^)). 

Then  the  model  of  the  m  radar  measurements  at  time  tj  Is 

m(t^)  ■  h(x(t1))  +  b  +  e(t1)  (8) 

Using  the  measurements  from  M  radars  over  a  sequence  of  N  times 
t^  the  potential  exists,  provided  the  relative  geometry  between  the 
object  trajectory  and  radar  sites  is  sufficiently  good,  to  estimate  both 

i 

the  position  vectors  x(tj),  1*1, N  and  the  measurement  bias  vector  b, 

Suppose  we  have  a  trial  or  guess  position  xQ(tj),  1*1  ,N.  Such  a  trial 

solution  Is  easily  computed.  For  example,  let  x0(t^)*(xu(t^) 

ya( ti )  7.a< ) )  be  the  cartesian  components  of  the  object  position  which 

can  be  computed  from  the  a**'  radar.  Then  a  good  guess  solution  Is 

obtained  from  xa^1^ 

x.(tj  *  median  yjtj  . 

•*■»[  w. 

Now  linearize  the  measurement  equation  (8)  about  xQ(t^), 

m( t,j )  «  h(xQ(t1 ))  +  H(xQ(t1))6x(t1)  +  b  +  e  (t^  (9) 


where  H(xQ(t^))  Is  the  3Mx3  matrix  of  partial  derivatives 


H(x(t1)) 


ah(x(t^) 

ax^) 


(10) 


Suppose  we  have 


A2 


estimates  a 


a2 


a2 


v  % 

a  a  a 


of  the  measurement  error 


variances  for  each  of  the  radars.  Let  l  be  a  diagonal  matrix  of 
these  variance  estimates.  Then  a  weighted  least  squares  estimate  of 
the  6x(t,| )  and  the  bias  vector  b  minimizes* 


r(t1 )  -  H(x0(t1))6x(ti)  -  bj 

i 


(id 


where  r(t^)  Is  the  residual  vector,  m(t^)  -  h(xQ(tj)). 

Rather  than  forming  the  least  squares  normal  equations  by  differentiating 
(11),  It  Is  more  accurate  and  convenient  to  solve  the  weighted  least 
squares  problem  using  the  QR  algorithm. 
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APPLICATION  OF  THE  QR  ALGORITHM  TO  THE  WEIGHTED  LEAST  SQUARES  PROBLEM 

In  order  to  cast  the  weighted  least  squares  problem  Into  an 
ordinary  least  squares  problem  to  which  the  QR  algorithm  can  be  applied, 
the  following  replacements  are  made 


r(ti )  *  l  “4ra(t.,) 

(12) 

H(x0(t1))  *  iMW) 

(13) 

e(t1 )  *  l  "4e(tj) 

(14) 

Then  the  weighted  least  squares  problem  posed  by  (11)  becomes  the 
ordinary  least  squares  problem  to  minimize 


N 


r(t1 )  -  H(xQ(ti ) )«Sx(t1 )  -  l  "*b 


(15) 


Thus,  at  each  time  t^  we  have  the  modified  measurement  equation 


r(t1 )  -  H(xn (t4 ) )6x(t^ )  -  l  “*b  +  e(ti ) 


o'  1 ' '  '  1 


1 


(16) 


Suppose  that  at  each  time  t^ ,  an  orthogonal  matrix  Q^  is  constructed 
such  that 


Q{H(x0(t1)) 


(17) 
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If  the  bias  vector  b  Is  known,  then  the  first  of  equations  (21) 
provides  the  least  squares  solution  for  the  Incremental  position 
vector,  6x(t^).  Thus,  6x ( t^ )  Is  obtained  by  solving  the  upper 
triangular  set  of  equations 

R1 6x(t1 )  -  yx(t1 )  -  Px{t1)b,  1*1, N 
The  second  of  equations  (21), 


r  v  v, 
'■\\v 

n  ,  •*  _  W 

v 


Pb(ti)b  "  W 


,  1-1, N 


provides  an  overdetermined  set  of  equations  to  be  solved  for  the 

bias  vector  b.  Let  Pb(ta)—  P„(tN)3.  yJ-Cy^tj)  yb(t2) — y„(tM)] . 

and  eb*[eb(t1)  e^) — efa(tN)].  Then 


Suppose  that  an  orthogonal  matrix  Qb  Is  constructed  such  that 


where  Rb  Is  a  3Mx3M  upper  triangular  matrix.  Then  the  least  squares 
solution  for  the  bias  vector  Is  the  solution  to, 


H 

1 


S 


m 


where  the  dimension  of  the  vector  Is  3M  and  the  dimension  of  Ug 

Is  N(3M-3)-3M.  Thus,  the  bias  estimate  Is  given  by  the  solution  to  the 
upper  triangular  system  of  equations, 

Rbb  -  Uj  (26) 

Having  computed  the  linearized  least  squares  estimate  of  the  bias 
vector  from  (26),  the  Incremental  position  solution  fix(t^)  Is  computed 
from  (22)  for  each  In  order  to  conserve  on  core  storage 
requirements,  the  QR  decomposition  In  (24)  Is  not  done  directly, 
since  Pb  Is  often  a  very  large  matrix.  Instead  of  a  direct  QR 
decomposition  on  P^,  the  QR  decomposition  is  done  sequentially  In  time 
as  each  Pb(t^)  arrives. 

Since  the  original  observation  equations  and  thus  the  original 
least  squares  estimation  problem  Is  nonlinear,  the  above  QR 
decomposition  process  for  the  least  squares  problem  must  be  Iterated 
with  the  replacement, 


After  the  iteration  process  has  converged,  new  estimates  of  the 

measurement  variances,  oD2,  o.2,  oca  are  computed  for  each  radar 

Ka  Aa  ta 

from  the  measurement  residuals, 


-  m(t1 )  -  h(xf(t1))  -  bf  (28) 

where  x^(t^)  is  the  estimate  of  the  position  vector  at  convergence, 
and  b|.  Is  the  estimate  of  the  bias  vector  at  convergence.  These  new 
estimates  of  measurement  error  variance  are  Inserted  Into  the  diagonal 

A 

variance  matrix,  ]iand  the  QR  iteration  process  again  Is  Iterated  until 
convergence.  This  outer  Iteration  loop  which  reestimates  the  measurement 
error  variance  Is  repeated  for  a  fixed  number  of  Iterations. 

Since  the  use  of  the  full  set  of  measurement  times  for  the 
estimation  of  the  measurement  biases  would  consume  considerable  computer 
time,  a  highly  thinned  set  of  measurement  times  Is  selected  for  use  In 
bias  estimation.  This  thinned  set  of  measurement  times  Is  selected 
to  cover  the  entire  object  trajectory.  This  allows  a  reliable 
estimate  of  the  measurement  biases  while  not  requiring  a  great  amount 
of  extra  computer  time. 


mK 


r„  ■% 
I  *•* 


3W 


SOME  EXAMPLES 


Although  there  have  not  been  any  convergence  problems  in  Iteratively 
solving  the  nonlinear  least  squares  problem  for  the  object  position  and 
the  radar  measurement  biases,  another  problem  which  sometimes  arises 
in  linear  least  squares  ‘sstimatlon  problems  occurs  frequently  in  the 
radar  bias  estimation  problem.  Very  often,  the  estimate  of  the  bias 
vector,  b,  converges  to  a  value  for  which  several  of  the  components  are 
too  large  and  may  have  the  wrong  sign.  Sometimes  the  bias  estimate  is 
obviously  erroneous.  One  obviously  erroneous  case  which  arises 
frequently  Is  that  the  elevation  bias  components  will  be  large  and 
of  the  same  sign.  In  linear  least  squares  estimation  the  problem 
of  the  vector  of  regression  coefficients  being  too  long  Is  often 
attributed  to  multi  col  linearity  among  the  predictor  variables.  This 
problem  In  the  linear  estimation  case  Is  often  successfully  treated  by 
some  method  of  biased  estimation  such  as  ridge  regression  or  principal 
components.  The  problem  has  not  been  properly  recognized  or  successfully 
treated  when  it  arises  In  trajectory  estimation.  Although  the  existence 
of  these  erroneous  bias  estimates  has  been  recognized  in  trajectory 
estimation,  the  source  of  the  difficulty  was  not  properly  recognized. 

Some  workers  in  trajectory  estimation  have  stated  that  the  existence 
of  this  problem  demonstrates  the  need  to  specify  a  prior  distribution 
for  the  biases  in  order  to  "tie  down"  or  statistically  constrain  the 
bias  estimates.  It  does  not  take  much  experience  in  using  these  priors 
for  trajectory  estimation  to  realize  that  the  problem  of  Inflated  bias 
estimates  is  as  much  present  with  the  prior  as  without  the  prior. 


We  have  attempted  to  treat  the  problem  of  inflated  measurement  bias 
estimates  with  both  ridge  regression  [1]  and  with  the  method  of 
principal  components  [2].  We  have  had  partial  success  with  each  of 
these  methods  but  neither  method  Is  satisfactory  for  an  automatic  data 
reduction  program. 

EXAMPLE  1: 

Consider  the  following  example  from  WSMR  tracking  data.  This 
example  has  three  radars,  R122,  R123,  R395  tracking  a  level  flying  drone 
flying  at  an  altitude  of  about  30,000  ft.  The  graph  of  Figure  1  shows 
the  relative  geometry  between  the  target  trajectory  and  radars.  The,, 
least  squares  estimates  of  the  radar  measurement  biases  for  this 
example  are, 


R122 

R123 

R395 

Range  bias  (ft) 

118.0 

115.1 

72.4 

Azimuth  bias  (mlllradlans) 

.186 

.143 

.  .179 

Elevation  bias  (mlllradlans) 

-.705 

-.911 

-.526 

The  values  of  the  elevation  bias  estimates,  which  are  all  large  and 
negative,  illustrate  a  common  type  of  erroneous  solution  occurring 
In  radar  trajectory  estimation.  In  this  example  we  are  able  to  confirm 
that  the  radar  bias  estimates  given  above  are  greatly  In  error. 

Using  measurements  from  tracking  cameras  we  are  able  to  obtain  position 
estimates  of  the  target  trajectory  which  are  considerably  more  accurate 
than  the  position  estimates  obtained  from  radar  measurements.  By  using 
the  optically  derived  positions  to  compute  what  the  radar  measurements 


should  have  been,  we  can  compute  the  actual  values  of  the  radar 
measurement  biases.  The  following  biases  were  computed  using  the 
optically  derived  positions. 


R122 

R123 

R395 

Range  bias  (ft) 

157.3 

152.9 

80.3 

Azimuth  bias  (mill radians) 

.05 

.02 

.09 

Elevation  bias  (ml  1 1  radians) 

.11 

-.08 

-.09 

The  large  errors  In  the  radar  bias  estimates  are  readily  apparent. 
EXAMPLE  2s 

This  example  has  three  radars,  R124,  R125,  R442  beacon  tracking  a 
high  performance  missile.  The  graph  of  Figure  2  shows  the  relative 


geometry  between  the  radars  and  object  trajectory. 

The  least  squares 

estimates  of  the  radar  biases  for  this  example  are 

R124. 

R125 

R442 

Range  bias  (ft) 

257.8 

304.8 

166.9 

Azimuth  bias  (mlllradlans) 

.056 

.049 

-.373 

Elevation  bias  (mlllradlans) 

-.170 

-.368 

-.479 

Radar  bias  estimates  derived  from  optical  measurements  are 


R124 

R125 

R442 

Range  bias  (ft) 

254 

297 

189 

Azimuth  bias  (mlllradlans) 

0 

-.04 

-.16 

Elevation  bias  (mlllradlans) 

0 

-.05 

-.09 
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Again  the  measurement  biases  estimated  from  least  squares  are 
seriously  In  error. 

VARIABLE  SELECTION  PROCEDURE 

The  stepwise  regression  procedure  In  the  BMDP  software  package  was 
tried  as  a  method  of  variable  selection  for  the  radar  measurement  bias 
estimation  problem.  Several  examples  were  tried  using  the  BMDP 
stepwise  regression.  Although  the  BMDP  procedure  proved  to  be  unsatisfactory 
for  many  of  the  examples,  the  results  suggested  changes  necessary  to  develop 
a  satisfactory  varlab'le  selection  procedure  for  the  bias  estimation  problem. 

•i 

The  computer  output  from  the  BMDP  stepwise  regression  routine  applied  to 
the  first  of  the  previous  two  examples  Is  presented  on  pages  433-436, 
Variables  x(l),  x(2),  and  x(3)  correspond  to  the  range  biases  for  R122, 

i 

R123,  R395  respectively.  Variables  labeled  x(4),  x(5),  and  x(6) 
correspond  to  the  azimuth  biases  for  R122,  R123,  and  R395.  Variables 
labeled  x(7),  x(8),  and  x(9)  correspond  to  the  elevation  biases  for  R122, 
R123,  and  R395.  The  stepwise  regression  for  this, example  uses  an  FJN  ■  6. 
Thus,  at  each  step  the  variable  with  the  largest  F-to-Enter  (provided  It  Is 
greater  than  FIN>  1s  entered  Into  the  radar  bias  model.  Thus,  In  step  11 
the  range  bias  for  R395  Is  entered  Into  the  model.  In  step  12  the  azimuth 
bias  for  R395  Is  entered.  In  step  #3  R122  elevation  bias  Is  entered  Into 
the  bias  model  and  In  step  #4  R122  range  bids  Is  entered.  In  step  15 
a  difficulty  occurs  In  the  BMDP  program.  At  this  point  variable  x(2) 

(R122  range  bias)  with  an  F-to-Enter  of  2102  should  have  been  entered  Into 
the  bias  model.  The  BMDP  output  Indicates  on  page  435  that  It  was  unable 
to  enter  x(2)  Into  the  model  because  to  do  so  would  lower  the  tolerance 


of  variable  x ( 1 )  below  Its  limit  of  .01.  This  Is  equivalent  to  saying 
that  this  bias  estimation  problem  Is  so  Ill-conditioned  that,  for  the 
numerical  method  being  used  by  BMDP,  the  minimum  conditions  for 
satisfactory  operation  cannot  be  met.  Finally,  the  variables  x(9) 
corresponding  to  R395  elevation  bias  and  x(4)  corresponding  to  R122 
azimuth  bias  are  entered  Into  the  radar  bias  model.  The  final  radar 
bias  estimates  In  the  model  are  given  on  page  436.  Comparing  these  bias 
estimates  with  the  radar  biases  derived  from  optical  measurements 
Indicates  that  the  bias  model  obtained  from  BMDP  stepwise  regression  Is 
erroneous.  The  reason  for  this  Is  the  failure  of  BMDP  to  enter  the 
variable  x(2)  corresponding  to  R123  range  bias.  The  value  of  the  R123 
range  bias  obtained  from  the  optical  measurements  Is  152.9  ft  which 
Indicates  that  It  must  be  a  significant  Influence  in  the  radar  bias 
model.  Thus,  this  example  suggests  that  a  better  numerical  method 
needs  to  be  used  for  the  current  application. 

The  computer  output  from  the  BMDP  stepwise  regression  routine  applied 
to  the  second  of  the  previous  two  examples  Is  presented  on  pages  437-44.1, 
As  In  the  previous  examples  variables  x(l),  x(2),  x (3)  correspond  to 
radar  range  biases  for  R124,  R125,  and  R442.  Variables  x(4),  x(5), 
x (6 )  correspond  to  azimuth  measurement  biases  for  R124,  R125,  and  R442. 
Variables  x(7),  x(8),  x(9)  correspond  to  elevation  measurement  biases  for 
R124,  R125,  and  R442.  In  this  example  of  stepwise  example  an  F|N»  1  Is 
being  used.  This  Is  done  so  that  all  variables  mSy  eventually  be  entered 
Into  the  model.  It  Is  Informative  In  this  example  to  observe  the 
sequence  of  multiple  Ra  values  as  variables  are  entered  Into  the 
measurement  bias  model.  After  having  entered  the  three  range  bias 
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variables,  x(l),  x(2),  x(3)  Into  the  model  the  multiple  R2  value  Is 
.9944  Indicating  that  99.44%  of  the  sum  of  squares  has  been  explained  by 
these  three  variables.  In  step  #4  x(6)  corresponding  to  R442  azimuth 
bias  Is  entered  and  results  In  a  multiple  R2  of  .9959,  an  Increase  of 
only  .0015  over  the  previous  value.  As  additional  variables  are  entered 
Into  the  model  the  sequence  of  Increases  of  multiple  R2  Is  .0003, 

.0004,  .0007,  .0001,  .0000.  Also,  as  these  additional  variables  are 
entered  Into  the  model,  the  magnitudes  of  these  additional  biases  are 
large  and  erroneous,  and  the  magnitude  of  x(6)  which  was  entered  In  a 
previous  step  also  becomes  Inflated  and  erroneous.  Thus,  variables 
x(4),  x(5),  x(7),  x(8) ,  and  x(9)  are  erroneous  and  are  Ineffective  in 
explaining  the  sum  of  squares.  This  example  suggests  that,  In  order  to 
combat  the  problem  of  erroneous  bias  estimates,  an  effective  model  building 
procedure  might  be  constructed  by  placing  a  lower  limit  on  the  amount  of 
change  In  the  multiple  R2  that  Is  acceptable  as  a  variable  Is  entered 


Into  the  model. 
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Both  a  forward  selection  procedure  and  a  backward  elimination  procedure 
were  developed  and  tested  as  radar  measurement  bias  model  builders.  At 
each  step  of  the  model  building  process  the  forward  selection  procedure 
enters  the  variable  Into  the  model  which  produces  the  largest  Increase 
In  the  multiple  Ra.  The  selection  process  Is  stopped  when  the  largest 
Increase  In  the  multiple  Ra  falls  below  a  specified  threshold.  6. 

Starting  from  the  model  which  Includes  all  of  the  radar  bias  parameters, 
at  each  step  of  the  model  building  process  the  backward  elimination 
procedure  deletes  the  variable  from  the  modql  which  produces  the 
smallest  decrease  In  the  multiple  R*.  The  elimination  process  Is 
stopped  when  the  smallest  decrease  In  the  multiple  R2  Is  greater  than  a 
specified  threshold.  6  or  when  the  error  sum  of  squares  has  Increased 
by  more  than  (1+f)  the  error  sum  of  squares  of  the  full  model.  The 
two  model  building  procedures  have  been  tested  on  numerous  data  sots. 
Sometimes  the  two  selection  methods  result  In  the  same  model,  but  more 
often  they  result  In  two  slightly  different  but  reasonable  measurement 
bias  models.  The  backward  elimination  model  builder  was  selected  for 
Implementation  In  the  WSMR  radar  reduction  program  because  It  allows 
one  to  obtain  some  Idea  of  the  effects  of  the  parameters  which  are  not 
In  the  final  model. 

Starting  with  the  full  bias  model  given  In  equation  (23), 


we  construct  the  orthogonal  such  that  Q^Pb  ■|^oDJ,  i*e*»  equation 
(24).  Then  the  measurement  bias  estimate  for  the  full  model  Is  given 
by  the  solution  to  equation  (25) » 


K1 

b  - 

M 

f - 

O 

N 

The  sum  of  squares  due  to  regression  for  the  full  model  Is 


SSR(B) 


U 


where  B  ■  {bj,  j-1,  3M)  denotes  the  full  model 


and  the  sum  of  squares  due  to  error  Is 

12 


SSE(B) 


U 


thus,  the  multiple  Rl  for  the  full  model  is 


(29) 


(30) 


Now  suppose  variable  bj  Is  deleted  from  the  full  model.  Let  Bj  denote  the 
model  with  bj  deleted,  Bj  ■  (b^,  k  +  j).  Then  the  regression  sum  of 
squares  with  bj  deleted  Is, 


SSR(Bj)  -  SSR(B)  -  SSR(bj|Bj) 


(32) 
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where  SSR(bj|8j)  Is  the  extra  sum  of  squares  due  to  entering  bj  Into 
the  model  given  that  all  other  variables  are  already  in  the  model.  We 
want  to  delete  from  the  full  model  the  variable,  bj  .  such  that  SSR(bj^  |Bj, ) 
Is  a  minimum  provided  SSR(bj^  |Bj)/(SSR(B)+SSE(B)<i6.  Thus,  we  choose 
bjy  such  that  SSR(bJ  jBJi)  <  SSR(bj  |Bj),  j»l,3M  and 

SSR(b.|  |Bj)/SSR(B)+SSE(B}<,£.  Choosing  the  minimum  of  SSR(bj|Bj)  Is 
equivalent  to  selecting  the  variable  bj  for  which  the  partial 
F-coefflclent, 


FJ  ""■SSEtffJ  ' 


(33) 


Is  a  minimum,  since  SSE(B)  Is  Independent  of  bj.  The  partial  F  Is 
easily  computed  as, 


(34) 


where  bj  Is  the  estimated  value  of  bj  In  the  full  model  and  ✓CJJ 
Is  the  standard  error  of  bj.  Thus  we  delete  the  variable  bj  from 
the  full  model,  where  F j ,  ■  min  {Fj,  J«l,  3M> .  The  j;—  column  of  the 
matrix  is  then  deleted  from  Pb  and  columns  +  1  thru  3M  are  moved  down 
one  column,  l.e.,  we  do  the  replacements  Pj  *  Pj+i *  3M-1  where 

Pj  Is  the  J—  column  of  P^.  The  backward  elimination  Is  then  repeated 
using  the  new  matrix  Pfa  which  has  only  3M-1  columns.  The  algorithm  for 
the  backward  elimination  model  building  process  Is  summarized; 
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*  I 


Ns  *  3M  -  STEP 


T  [< 

1.  Construct  an  orthogonal  Qb  such  that  QbPb  ■  °  , 


where  Is 


3.  Solve  Rfab  ■  Qbyb  ■  ^dimension  {b}  *  Ns 


3.  Compute  SSR  ■  pl||\  SST  -||li||a  ,  R*«  ||lji 


4.  Compute  C  ■ 

b, 

5.  Compute  partial  F's*  Fj  *  ,  j-l»Ns 

J  J 


6.  Fj i  ■  min  {Fj,  J-l,  Ns) 

A 

If  F4  /SST  >5,  stop  and  use  b  as  the  final  model, 
Jj, 


STEP-0, 

3M-1 


else  make  the  column  replacements,  pb(1)*pb(1+l),1-^',N8-l,J 
An  alternative  for  stopping  the  elimination  process  Is  to  stop  at  step 
Jk-1  when  SSE(Bj  j...j^)>(l+f)SSE(B)  where  Bj  j  ■ 

(bj,  1  f  J1#  and  f  Is  a  small  fraction,  say  f  *  .1. 


The  deleted  variables,  bj  ,  b.  — b,  ,  are  considered  as  dependent 

variables  and  the  set  of  estimated  variables  B<  <  — - *  are 

J1J2  Jk-i 

considered  as  Independent  variables.  Having  Identified  and  estimated 
the  Independent  variables,  values  for  the  dependent  variables  are 
computed  as  averages  of  residuals.  Let  9  1f*jx  ,  j2 , — J k-i  be  th* 
estimated  values  of  the  Independent  variables.  Let  bj  be  the  vector  with 
components  b^  for  IrtJj,  Ja »— -Jk  and  0  for  the  components 
corresponding  to  the  dependent  variables.  Let  m’(t.|)  be  the  observation 

A 

vector  at  time  t^  with  the  vector  bj  removed, 


m'(t,|)  ■  m{tj)  -  bj  ■  htf^))  +  (b  -  bj)  +  e(t1) 


:  (35) 


Linearization  and  the  QR  algorithm  are  again  applied  to  computing  the 

i 

position  vectors  Su  1-1 ,N  which  minimize 

•  jjK) -  mhjf  ( 

lf 

where  Is  the  final  estimated  covariance  matrix  computed  from  the 
measurement  residuals,  (28).  The  values  of  the  dependent  variables 

A 

b1’  • — J k.i  are  now  computed  as 

A 

where  x(t^)  Is  the  estimated  position  vector  at  tlme^t^  obtained  by 
minimizing  (36). 
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EXAMPLE  1: 

When  the  backward  elimination  variable  selection  technique  described 
In  the  last  section  Is  applied  to  the  first  of  the  two  examples  previously 
described,  the  following  results  are  obtained. 


Full  Model 

Estimated  biases 


122-R 

122-A 

122-E 

123-R 

123-A 

123-E 

395-R 

395-A 

395-E 

118.0 

.196 

-.705 

115.1 

.143 

-.911 

72.4 

.179 

-.526 

SSR  « 

18972,  SSE 

-  547 

Ra  - 

.9810 

Partial 

F's 

161.8 

3.0 

3.98 

154.4 

1.60 

6.67 

39.0 

13.0 

7.27 

123-A 

deleted 

Step  1  bias  estimates 

122-R 

122-A 

122-E 

123-R 

123-E 

395-R 

395-A 

395-E 

118.0 

.053 

-.779 

115.1 

-.985 . 

59.1 

.118 

.559 

SSR  ■ 

18971,  SSE 

-  549 

Ra  - 

.9810 

Partial 

F's 

162.0 

57.3 

ED 

154.5 

8.0 

149.5 

91.0 

8.9 

122-E 

deleted 

Step  2  bias  estimates 

2-A  123-R  123-E  395-R  395-A 

053  135.0  -.207  69.0  .134 

SSR  -  18966t  SSH  -  554  Ra  -  .9807 

Partial  F*s 

2756.1  |S7.ll  2632.5  1291.2  1278.4  173.3 

122-A  deleted 

Step  3  bias  estimates 


SSR  -  18908,  SSE  -  610  Ra  -  .9777 

Partial  F‘s 

2753.5  2629.9  1291.2  1271.2  |l72.lj  204 

395-A  deleted 

Step  4  bias  estimates 


395-E 

-.140 


SSR  -  18736,  SSE  -  782 

Ra  ■  .9688 

Partial  F's 

395-E  de  eted 


122-R  123-R  123-E  395-R 


Independent  bias  variables 


162.1  159.1  -.205  86.4 

SSR  «  18561,  SSE  -  958  R2  -  .9509 
Partial  F's 

6176.1  5927.1  1264.3  3197.9 

Further  deletion  of  variables  would  cause  more  significant  decreases 
in  R2.  The  dependent  variables  are  the  122-A,  122-E,  123-A,  395-A, 
and  395-E  biases.  These  biases  are  computed  from  the  corresponding 
residuals.  The  results  are 


122-A 

122-E 

123-A 

395-A 

395-E 

.057 

.001 

-.002 

.075 

-.127 

-  dependent  bias  variables 


These  values  of  the  radar  bias  estimates  compare  favorably  with  the 
values  computed  from  optics, 

122-R  ’,22-A  122-E  123-R  123-A  123-E  395-R  395-A  395 


ni 


,ii 


ill 
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EXAMPLE  2: 


The  following  results  are  obtained  when  the  backward  elimination 
variable  selection  technique  Is  applied  to  the  second  of  the  two 
examples  previously  described. 


Full  Model 
Estimated  Biases 


124-R 

124-A 

124-E 

125-R  125-A 

125-E 

442-R 

442-A 

257.7 

SSR  - 

.056 

17867,  SSE 

-.170 

-  339 

304.8  .049 

R2  ■  .9813 

Partial  F's 

-.368 

166.9 

-.373 

5537.0  2.0 

125-A  deleted 

28.1 

4579.0  0.6  32.3 

Step  1  bias  estimates 

661.9 

25.0 

124-R 

124-A 

124-E 

125-R  125-E 

442-R 

442-A 

442-E 

258.6 

SSR  - 

.046 

17866,  SSE 

-.167 

-  339 

304.2  -.360 

R2  *  .9813 

Partial  F's 

167.6 

-.344 

-.472 

6159<0  1  I 

124-A  deleted 

27.5 

4672.1  31.7  679.9 

Step  2  bias  estimates 

28.2 

52.8 

124-R 

124-E 

125-R 

125-E  442-R 

442-A 

442-E 

257.5 

SSR  ■ 

-.158 

17865,  SSE 

301.5 

■  341 

-.337  170.4 

R2  «  .9813 

Partial  F's 

-.321 

-.461 

6602 

124- E 

k  - 

26.0 

deleted 

6084.2 

30.4  804.2 

26.8 

51.4 

442 -E 
-.479 


53.4 
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Step  3  bias  estimates 


124-R 

125-R 

125-E 

442-R  442-A 

442-E 

256.1 

300.2 

-.141 

178.9  -.296 

-.212 

SSR  ■ 

17839,  SSE 

«  367 

R2  ■  .9798 

Partial  F's 

6580.2  6058.3 

12B-E  deleted 

8.8 

951.0  22.9  25.7 

Step  4  bias  estimates 

124-R 

125-R 

442-R 

442-A  442-E 

255.1 

298.8 

181.7 

-.289  -.154 

SSR  - 

17830,  SSE 

-  376 

R2  ■  *9793 

Partial  F‘s 

6604.3  6090.2 

442-E  deleted 

1018.2 

21.8  17.3 

Step  5  bias  estimates 

124-R 

125-R 

442-R 

442-A 

260.4 

305.0 

178.8 

-.263 

SSR  - 

17813,  SSE 

-  393 

R*  ■  .9784 

Partial  F's 

8230.9 

7456.9 

1001.0 

noi 

442-A  deleted 
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Step  6  bias  estimates 


-  Independent  variables 


R2  -  .9773 

Partial  F's 

10411.9  7496.6  988.0 

Additional  deletion  of  variables  would  produce  large  decreases  In  Rz. 

The  dependent  variables  are  Identified  as  the  124-A,  124-E,  126-A,  125-E, 
442-A,  442-E  bias.  The  values  of  these  biases  computed  from  the 
residuals  are, 

124-A  124-E  126-A  125-E  442-A  442-E 

.020  .002  -.105  -.048  -.180  -.016 

These  values  of  the  radar  bias  estimates  again  compare  favorably  with  the 
values  computed  from  optics, 

124-R  124-A  124-E  125-R  125-A  125-E  442-R  442-A  442-E 

254  0  0  297  -.04  -.05  189  -.16  -.09 


124-R 

125-R 

442-R 

265.5 

305.5 

177.3 

SSR  “  17794,  SSE  ■  411 
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The  backward  elimination  variable  selection  technique  described 
above  has  been  successfully  applied  to  numerous  other  examples  of 
radar  bias  estimation.  A  question  in  the  application  of  this  method 
which  has  not  been  totally  answered  Is  wher.  to  stop  the  elimination 
process.  From  all  applications  tried,  It  appears  that  It  Is  best 
to  stop  the  elimination  when  the  fractional  change  In  the  SSE 
exceeds  a  given  threshold.  For  these  applications  a  plot  of  the 
SSE  versus  the  elimination  step  number  yields  a  curve  whose  general 
characteristics  are  shown  In  the  following  graph. 


I - 1 - ( - 1— . <  — — t . 4 . 

0  1  2  3  4  5  6  7 

STEP  NUMBER 


One  would  expect  that  a  good  stopping  point  Is  close  to  the  knee  of 
this  curve,  probably  Just  past  the  knee.  In  some  applications  It  has 
been  advantageous  to  go  two  points  past  the  knee.  A  good  answer 
about  when  to  stop  will  probably  come  with  additional  application 
experience. 
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ON  THE  LEHMANN  POWER  ANALYSIS  FOR  THE  WILCOXQN  RANK  SUM  TEST 

James  R.  Knaub,  Jr. 

US  Army  Logistics  Center 

ABSTRACT 

The  Wllcoxon  Rank  Sum  (or  Mann-Whltney)  test  Is  among  the  most  useful 
and  powerful  of  tho  non-parametric  hypothesis  tests.  However,  as  with  many 
hypothesis  tests,  when  a  clear  alternative  hypothesis  and  corresponding 
power  analysis  Is  not  present,  the  practical  Interpretation  of  results 
using  this  test  suffers  greatly.  This  paper  presents  and  clarifies  an 
alternative  suggestd  by  E.  L-.  Lehmann  In  19S3  and  provides  tables  of 
practical  use  which' have  not  prvlously  been  calculated  due  to  computational 
difficulties. 
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On  the  Lehmann  Power  Analysis  for  the 
Wllcoxon  Rank  Sum  Test 

The  Wllcoxon  Rank  Sum  (or  Mann-Whltney)  Test  Is  among  the  most  useful  and 
powerful  of  the  non-parametrlc  hypothesis  tests,  However,  as  with  many  hypo¬ 
thesis  tests,  when  a  clear  alternative  hypothesis  and  corresponding  power 
analysis  Is  not  present,  the  practical  Interpretation  of  results  using  this 
test  suffers  greatly,  This  paper  presents  and  clarifies  an  alternative  sug¬ 
gested  by  E.  L  Lehmann  In  1953  (Annals  of  Mathematical  Statistics  [7])  and 
provides  tables  of  practical  use  which  have  not  previously  been  calculated  due 
to  computational  difficulties.  This  work  has  recently  been  applied  to  survey 
data  gathered  for  the  US  Army  Logistics  Canter,  (See  reference  [5],) 

When  sample  sizes  are  small,  and  a  power  analysis  Is  not  available,  one  may 
fall  to  reject  the  null  hypothesis  when  the  true  state  of  nature  Is  very 
different  from  what  Is  stated  In  the  null  hypothesis,  With  a  small  sample  size 
and  smaller,  It  may  be  Impossible  to  reject  H0.  Further,  when  sample  sizes  are 

very  large,  the  null  hypothesis  may  be  rejected  at  a  very  small  significance 
level  when  actually  the  null  hypothesis  is  so  nearly  true,  that  It  Is  close 
enough  for  all  practical  purposes.  Taken  to  the  extreme,  with  Infinite  sample 
sizes,  the  attained  significance  level  will  be  zero,  even  when  there  Is  only  a 
vary  small,  but  finite  difference  between  HQ  and  the  true  state  of  nature. 

Thus  significance  level  can  be  very  misleading  If  used  alone, 

•  i 

When  a  null  and  a  definitive  alternative  hypothesis  can  both  be  stated,  and 
probability  distributions  found  under  each,  the  results  of  an  hypothesis  test 
can  be  stated  similarly  to  a  confidence  Interval  If  the  “point  estimate11  from 
the  observed  values  falls  between  the  two  hypotheses.  In  the  case  of  the 
Wllcoxon  Rank  Sum  Test,  only  one  alternative  hypothesis  has  been  well  developed 
and  will  be  presented  here,  Oue  to  the  nature  of  this  test,  however,  even  If 
the  evidence  may  strongly  Indicate  that  the  true  state  of  nature  Is  not  bounded 
between  this  alternative  and  the  null  hypothesis,  this  power  analysis  can  still 
be  used  to  obtain  a  reasonable  estimate  of  what  the  actual  state  of  nature 
happens  to  be,  (In  the  case  of  the  Multiple-sample  Westenberg-type  tests  of 
reference  [4],  an  alternative  must  be  picked  sucn  that  the  true  state  of  nature 
is  Indicated  to  be  bounded  by  the  null  end  alternative  hypotheses.  Fortun-  , 
ately,  that  Is  not  the  case  here,  nor  was  It  the  case  In  reference  [6],  which 
Is  a  multi-sample  test.) 

Consider  that  the  null  hypothesis,  HQ,  of  the  Wllcoxon  Rank  Sum  Test 

Indicates  that  P(X<Y)  ■  1/2.  That  Is,  under  HQ,  any  value  picked  at  random 

from  the  Y  population,  Is  larger  than  any  value  picked  at  random  from  the  X 
population,  with  probability  of  1/2.  Here  an  alternative  hypothesis,  Is 

used  such  that  P(X<Y)  ■  2/3.  (The  exact  form  of  Hj  Is  discussed  In  [7],) 

Graph  1  Illustrates  a  possible  configuration  for  this  alternative  hypothesis. 
For  this  example,  consider  that  under  HQ,  all  observations  are  taken  from  a 

N(r,s)  distribution  such  as  the  N(5,l)  shown  on  the  left  in  graph  1,  but  under 
Hj,  the  Y  sample  comas  from  the  N(r+Q.61s,  s)  distribution,  while  the  X  sample 

comes  from  the  N(r,s)  distribution. 


Another  example  of  a  possible  situation  satisfying  the  alternative  hypo¬ 
thesis,  Hj,  given  approximately  by  comparing  a  gamma  (4,1)  with  a  gamma  (3,1), 

Is  Illustrated  by  graph  2. 

Hote  that  the  Wilcoxon  Rank  Sum  Test  Is  most  sensitive  to  location,  a 
little  sensitive  to  shape,  but  not  to  dispersion  (except  as  ft  relates  propor¬ 
tionately  to  differences  In  location).  Therefore,  it  Is  the  differences  In 
’ocatlon  that  are  of  primary  Importance  In  graphs  1  and  2. 

In  order  to  determine  the  probability  of  drawing  a  value  from  distribution 
A  which  Is  larger  than  a  simultaneously  drawn  value  from  distribution  B,  the 
following  may  be  used: 

p  ■  f”ffl(t)dtdx 

x*-«  t«x' 

where  and  fg  represent  density  functions, 

For  the  cose  where  A  and  B  are  both  gamma  distributions, 
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For  gamma  (4,1)  and  gamma  (3,1),  p  ■  21/32  *  0.65b. 

For  normal  distributions,  usd  -  Mp)//o|  +  ojj]  t  as  In  tho  Church-Marris- 

Oownton  (C-H-D)  method  of  missile  motor  satet>  testing  [2].  (Note:  This 
reference  to  the  C-H-Q  method  should  not  be  construed  as  the  author's  endorse¬ 
ment  of  this  method  for  the  purpose  of  missile  motor  safety  testing.) 

The  calculation  of  power  under  this  alternative  Involves  a  summation  over  a 
typically  large  number  of  products.  Calculation  of  this  value  can  become 
extremely  time  consuming,  even  for  a  high  speed  computer.  A  program  was 
written  for  the  author  at  White  Sands  Missile  Range  which  will  calculate  these 

exact  values,  however,  In  general,  the  sample  sizes  must  be  very  small. 

Recently,  however,  the  author  constructed  a  simulation  which  provides  estimates 
of  the  power  for  much  larger  sample  sizes.  A  number  of  the  "products"  men¬ 
tioned  earlier  are  calculated  and  the  mean  Is  computed.  1  he  number  of  products 
Involved  In  the  exact  calculation  can  be  determined,  and  It  Is  multiplied  by 
this  mean.  Comparison  to  values  calculated  exactly  (when  practical),  and  a 
study  of  the  sensitivity  of  the  results  t.o  increased  replications,  as  well  as 
comparison  to  other  simulated  values  bounding  the  results  In  the  tables,  led  to 
the  use  of  from  I  to  20  million  replications  to  simulate  values  for  the  tables 
found  In  this  paper.  (Work  has  bean  done,  reference  [3],  to  determine  the 
number  of  simulation  replications  needed  under  less  radical  circumstances. 

Here,  however,  a  larger  number  of  replications  appears  nece'Suiy.)  (For  n  »  in 


B  50,  up  to  35  million  replications  were  used.  It  appeared,  however,  that 
fewer  replications  using  a  number  of  different  seeds  yielded  mean  answers  which 
more  quickly  converged  to  reasonable  results,  especially  when  using  antithetic 
seeds,) 

In  the  tables,  n  Is  the  sample  size  of  the  X  sample,  m  Is  the  samplt  size 
of  the  Y  sample,  RS  is  the  rank  sum  for  which  type  I  and  type  II  error  proba¬ 
bilities  are  calculated,  PA  Is  the  former  of  those  probabilities,  and  PB  Is  the 
later.  Specifically,  PA  Is  the  attained  probability  of  making  an  error  If  KQ 

Is  rejected,  and  PB  Is  the  attained  probability  of  error  If  Hj  Is  rejected, 

both  corresponding  to  the  same  RS  value,  RS  Is  always  calculated  by  adding  the 
ranks  of  the  Y  elements  In  the  combined  sample.  Note  that  for  smaller  sample 

sizes,  power  +PB  Is  noticeably  larger  than  unity  due  to  the  discrete  nature  of 

this  test.  That  Is,  the  probability  of  obtaining  exactly  the  event  observed 
(and  no  other)  Is  non-zero. 

Three  significant  digits  are  given  for  PA  and  only  two  for  power  and  PB 
simply  because  It  takes  fewer  replications  of  the  simulation  to  satisfactorily 
obtain  a  value  for  PA  than  for  the  others. 

From  the  annex  to  table  1,  It  Is  found  empirically  that  If  x  is  the  size  of 

each  of  the  two  samples,  and  f  (x)  Is  the  probability  of  a  type  II  error 
under  the  alternative  used  here,  adjusted  to  correspond  to  a  specific  signif¬ 
icance  level,  thon,  as  a'contlnuous  representation  of  actually  a  discrete  process, 

fn.inM  “  e*P(-x/16) 

for  at  least  3  <  x  <  40,  and  perhaps  this  approximation  could 
be  trusted  for  x  ■  45  or  larger.  “However,  extrapolations  are  always  more 
dangerous  than  Interpolations,  so  caution  Is  advised  for  further  extensions. 

For  a  ■  0.05^ 


f0,ns^)  “  exp(-x/[26exp  ^]) 

for  at  least  4  <  x  <  40,  and  perhaps  for  x  substantially  larger.  Using  this 
approximation,  Tt  Is  conjectured  that  for  n  ■  m  *  66,  when  PA  Is  approximately 
0.05  (RS  ■  4751),  then  PB  for  this  alternative  Is  also  approximately  0.05  and 
the  true  state  of  nature  would  then  quite  safely  be  said  to  (probably)  lie 
between  the  null  and  alternative  hypotheses.  (At  the  0.1  probability  level  for 
PA  and  PB,  this  could  be  said  When  n  ■  m  ■  37,  and  RS  »  1507.)  An  extrapola¬ 
tion  to  n  ■  m  ■  66  Is  questionable,  however,  and  further  extrapolation  Is  not 
advised.  Computer  simulation  for  n  *  m  ■  50  Indicates  that  for  the  top  curve 
(PA  :  0.05)  In  Annex  I  to  table  1,  true  values  In  this  area  for  PB  may  be 
somewhat  smaller  than  this  curve  predicts.  For  PA  :  0,10,  PB  values  for  large 
n  and  tn  may  be  somewhat  larger  than  predicted. 


m 
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In  Conover's  book  NT.  an  approximation  Is  given  to  find  RS  for  a  given  PA 

value.  (RS  *;  m(m  +  n  +  1)/H  +  x^B^n(n  +  in  +  1  )/12  .  where  x1-a  Is  from 

the  table  of  the  cumulative  normal  distribution.)  The  two  functions  given 
earlier  can  be  used  to  estimate  PB  values  when  PA  «  0.10  or  0.05. 

The  final  graphs,  3-7,  are  taken  from  work  ths  author  directed  at  White 

Sands  Missile  Range  In  order  to  study  this  alternative  for  the  Wllcoxon  Rank 
Sum  Test  with  emphasis  on  simulation  validation  for  missile  flight  simulations. 
When  comparing  a  very  few  live  firings  to  a  substantially  larger  number  of 
simulations  for  each  scenario,  It  can  be  seen  from  these  graphs  that  once  one 
sample  Is  substantially  larger  than  the  other,  Increasing  the  larger  sample 
size  further  does  very  little  to  Improve  the  power.  These  graphs  are  contin¬ 
uous  representations  of  what  are  actually  discrete  points.  The  values  for 
those  points  were  calculated  analytically  as  noted  In  the  acknowledgements. 

.  Finally,  when  n/m,  PB  can  be  bounded  using  the  exponential  formulations 
found  earlier  In  this  paper.  If,  for  example,  RS  Is  such  that  PA  ■  0.1,  and 
x-|  ,  Is.  the  smaller  of  n  and  m,  and  x2  Is  the  larger,  then  one  has  that  approx¬ 
imately  exp(-x2/16)  <  PB  <  expf-x^,  with  PB  somewhat  closer  to 

expi-x^lB),  especially  when  x1  «  x2. 

For  larger  sample  sizes  than  are  handled  here,  parametric  methods  may  be 
used.  However,  In  addition  to  the  probability  of  error  associated  with  any 
conclusion  drawn  from  a  parametric  test,  there  Is  the  additional  risk  Involved 
In  assuming  the  distributional  forms  use  m  such  a  test.  Hypothesis  tests 
should  also  be  used  to  study  these  distributional  assumptions  to  provide  a  more 
complete  risk  analysis. 

EXAMPLE:  . 

Consider  two  sources  of  data,  X  and  Y,  where  It  Is  suspected  that  Y  may 
represent  a  population  of  larger  location  than  X,  but  this  Is  not  clear.  If  11 
observations  are  taken  from  the  X  population,  and  19  observations  taken  from  Y, 
then  the  critical  value  of  the  rank  sum  (RS)  of  the  Y  sample  observations 
within  the  combined  sample  which  represents  the  point  at  which  rejection  of  the 
null  hypothesis  would  occur  using  a  ■  0.10,  Is  approximately 


RS 


in (m  +  n  +  l)/2  +  1.2816/iiinTm '+  n"  +  ))/i2 


■ 


(lP)(31)/2  +  1. 


:  324.3 


Therefore,  If  RS  >  325,  HQ  would  be  rejected  at  the  «  a  0.10  level.  However, 
should  RS  ®  325,  and  HQ  not  be  rejected,  then  the  probability  of  making  a  type 
II  error  with  respect  to  the  alternative  hypothesis  Illustrated  In  graphs  1  and 
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2  Is  approximately  boundad  by  axp  (-19/16)  and  exp  (-11/16),  so  0.30<PB<0.50. 
Note  that,  from  table  2  ,  when  PA  ■  0,099,  PB  (10,20)  s  0.43.  Using  4,000,000 
replications  In  the  program  given  In  Appondlx  A,  for  m  *  19,  n  ■  11,  and  RS  » 
325,  resulted  In  PA  *»  0.100  and  PB  ■  0.42. 
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ADDENDUM 


Multiple  applications  of  this  test  can  be  used  to  compare  two  levels  of  a 
factor  under  a  numbor  of  conditions.  If,  for  example,  manufacturer  A  produces 
a  machine  which  Is  suspected  to  have  higher  reliability  under  most  scenarios 
than  a  similar  machine  made  by  manufacturer  B,  then  under  each  of  the  y 

scenarios,  m^  Is  the  sample  size  of  A's  machines  and  n^  is  the  sample  size 

of  B's  machines,  for  1  *  1  to  Y-,  PAj  and  PB^can  be  calculated  for  each  of  the 

scenarios.  Consider  0  <  a  <  Y  and  0  <  b  <  Yi 

PA  Is  the  probability  of  a  or  more  PA^'s  being  less  than  pA 

(1  «  1; Y  j,  when  HQ  Is  true. 

P8  Is  the  probability  of  b  or  more  PB^'s  being  less  than  pg 


Therefore, 


(1  *  1,  Y  ),  when  Is  true. 


.nd  M  (5lP*0  'PftlY’X 

PB  ■  jb  (X)Pg(1  -  Pb)y‘X 

are  chosen  to  be  reasonable  considering  sample  sizes  for  each  of  the  y 


If  ■  1  then  the  evidence  shows  that,  In  general,  the  true  state  of  nature 
Is  just  as  likely  to  be  equivalent  to  as  HQ. 

If  a  2  then  the  evidence  Indicates  that,  in  general,  the  true  state  of 

nature  Is  twice  as  likely  to  be  equivalent  to  HQ  as  Hj.  If  PA  and 

PB  are  small,  then  the  Indication  Is  only  that  the  true  state  of 
nature  Is  closer  to  HQ  than  H^,  although  possibly  not  very  close 

to  either. 


(Nbte  that  another  paper  In  this  conference,  "Numerical  Validation  of 
Tukey's  Criteria  for  Clinical  Trials  and  Sequential  Testing,"  by  C.  n.  lenko, 
also  deals  with  this  type  of  problem,  and  was  of  Interest  to  this  author.) 
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At  this  time,  this  methodology  Is  being  used  to  determine  whether  survey  data 
from  a  presumably  less  reliable  source  Is  compatible  with  a  presumably  superior 
data  source.  Difficult  to  obtain  drta  on  U.S.  Army  warehousing  activities  have, 
as  one  obvious  characteristic,  a  Vb*y  flat  "peak*"  Therefore,  a  sample  median 
value  can  be  changed  drastically  by  the  addition  or  deletion  of  one  data  point. 

If  the  secondary  data  source  proves  to  provide  values  distributed  closely 
enough  to  that, of  the  primary  source,  the  advantaqe  of  Including  this  source 
may  outweigh  the  disadvantage.  The  current  situation  Is  more  complex 
than  this.  However,  some  results  employing  the  methodology  of  this  addendum, 
have  been  realized. 


ADDENDUM  2 

Two  approximations  for  the  power  of  this  test  which  apparently  are  good 
for  a  wide  range  of  normal  alternative  hypotheses  are  to  be  found  In  • 

E.  L.  Lehmann,  Nonparametrlcs;  Statistical  Methods  Based  on  Ranks,  Holden-Day, 
1975.  A1 though  restricted- to  normal  alterna 1 1 ves  In  the  forma  tin  which  they 
are  written,  these  approximations  can  be  used  to  extend  the  tables  given  here 
to  larger  n  and  m.  The  easier  of  the  two  approximations  to  apply,  In  Its 
simplest  form,  Is  found  on  page  73  of  the  above  reference  and  Is  essentially  as 
follows: 

-  . .  /  3mn  yA _ u  i 

power  *  (m'+“rTTT57  a  X1~«J 

where  in  our  case  we  have  (u^  -  U^)/®  “  0.610. 

Note  that  in  the  example  in  the  main  body  of  this  paper  (m  ■  19, n  “11), 
that  this  approximation  gives  power  "  0,60,  which  is  consistent  with  what 
was  shown  earlier. 
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1.  Uncertainty,  Validity,  and  Stringency. 

Some  experimentation  needs  to  be  done  with  the  least  uncertainty  possible.  Some  of 
this  is  devoted  toward  direction  of  change,  and  can  thus  be  analyzed  in  terms  of  a  simple 
significance  test  or  a  simple  confidence  interval  -  perhaps  obtained  by  trial  and  error  bisec* 
tion  among  displaced  significance  tests. 

In  such  cases  we  need  to  reduce  all  kinds  of  uncertainty,  including  uncertainty  about 
any  statistical  hypotheses  that  matter  for  our  analysis. 

For  a  simple  significance  test  to  be  valid,  we  must  control  the  chance  of  significance  in 
the  null  situation  -  allowing  it  to  be  at  most  inappreciably  greater  than  the  announced  P* 
value. 


We  would  like  the  null  situation  to  be  as  broad  as  possible,  since  the  broader  it  is,  the 
more  usefully  valid  is  our  analysis.  The  broadest  null  situation  that  we  know  how  to  handle 
well ,  when  we  have  treatment  and  control  in  blocks  of  two,  is: 


Any  set  of  observed  results  is  as  likely  to  occur  for  any  interchanges,  each  within  a 
block,  of  treatment  with  control  -  "all  reassignments  are  equally  null*probable". 

Notice  that  no  assumption  is  made  about  relative  likelihoods  of  " different "  sets  of 
observed  results.  There  is  no  "statistical  model"  in  the  ordinary  sense,  and  there  is  thus  no 
need  to  worry  how  well  -  or  how  poorly  -  such  a  model  approximates  the  real  world. 
When  we  are  being  careful,  we  should  know  why  our  analysis  is  valid!  We  should  not 
depend  on  unverlfiable  assumptions  for  validity. 

Validity  b  of  course  not  enough.  We  surely  do  not  went  to  be  wasteful.  So  we  must 
also  seek  ttringency  as  expressed  in  such  technical  terms  as 


minimized  variance 
actual  variance 


efficiency 


minimized  typical  length  of  confidence  interval  \ 
actual  typical  length  of  confidence  interval  J 


Here  "variance"  or  "typical  length",  while  hopefully  imitating  what  goes  on  in  the  real 
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Keynote  Addreo,  30th  Conference  on  Detign  of  Experiment!,  Army  Retareh,  Development  and  Teiting,  La*  Cruet*, 
New  Mexico,  October  17, 1984 
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world,  can  only  be  assessed  in  terms  of  (i)  a  statistical  model  or  (ii)  sampling  from  a  larger 
body  (probably  bodies,  since  one  body  is  unlikely  to  suffice)  of  real  dais.  Assessment  in 
terms  of  a  statistical  model  can  be  either  by  formula-manipulation  or  by  numerical  calcula¬ 
tion,  which  will  usually  require  some  form  of  experimental  sampling  (direct,  Monte-Carlo, 
configural,  etc.). 

If  we  could  gain  stringency  without  fixing  a  statistical  model  -  or  class  of  statistical 
models  -  our  desire  for  certainty  would  drive  us  to  do  so.  But  we  do  know  that  pairs  of  sta¬ 
tistical  models  can  be  invented  to  be  so  antithetic  to  one  another  that  stringency  for  either 
rules  out  stringency  for  the  other.  The  best  certainty  we  can  reach,  therefore,  comes  by 
using  a  fairly  flexible  clait  of  statistical  models  -  one  which  we  hope  parallels  the  real  world 
in  whatever  ways  are  most  important. 

This  OFTEN  means  that  we  must  seek  robustness  of  ttringency. 

During  the  last  decade  or  so  much  progress  has  been  made  in  identifying  summaries  of 
small  and  moderate-sized  samples  that  do  provide  robustness  of  stringency,  but  little  atten¬ 
tion  has  been  paid  to  how  such  summaries  should  be  built  into  experimentation. 

We  can  have  both  high  validity  and  high  robustness  of  stringency.  When  we  are  being 
careful  we  SHOULD  arrange  for  BOTH. 

And  if,  in  special  circumstances,  some  ether  attainable  property  is  more  important  than 
robustness,  we  can  usually  combine  it  with  guaranteed  validity  by  selection  of  a  different 
kind  of  summary.  When  we  wish  to  be  careful,  we  should  arrange  for  both  validity  and  pre¬ 
ferred  spedally-useful-property. 

2.  The  Fishtr-Babbage  "legacy  of  a  legacy". 

Sir  Ronald  Fisher  and  Charles  Babbage  left  us  two  things,  which  together  were  not 
then  a  legacy,  but  could  -  and  have  -  grown  up  to  become  one.  Namely  randomized  exper¬ 
iments  and  computers. 

3.  The  economics  of  computation  has  changed;  explicit  reassignment  is  feasible  and 
needed! 

It  was  almost  50  years  ago  that  we  heard  -  with  respect  and  perhaps  a  touch  of  awe  ~ 
that  L.J.  Comrie  had  half  a  million  multiplication  to  do,  and  had  rented  a  set  of  punched- 
cards  machines  (what  are  now  dignified  as  "unit  record  equipment")  for  tix  months  to  do 
just  this. 

This  year,  one  can  buy  ~  for  no  more  than  a  statistician's  annual  salary  -  a  workstation 
with  a  few  megaflops  of  capacity.  One  which,  since  it  can  do  millions  of  multiplications  in  a 
second,  could  (I/O  aside)  do  Comrie's  6-month  chore  in  a  fraction  of  a  second. 

The  Important  conclusions  are  these: 

e  If  a  million  arithmetic  operations  can  save  a  second's  time  for  a  statistician  -  and  the 
opportunity  recurs  often  encough  -  we  cannot  afford  NOT  to  do  the  arithmetic. 

«  If  reanalyzing  the  observed  data  according  to  a  few  thousand  alternative  assignments 
(of  treatment  and  control)  let  us  increase  the  certainty  of  our  analysis  enough  to  save 
either  the  statistician  or  his  client  on  second  of  worry,  we  cannot  afford  not  to  do  the 
few  thousand  reanalyses. 

•  If  reanalyzing  the  observed  data  according  to  a  few  thousand  alternative  assignments  let 
us  provide  more  robustness  of  stringency  for  our  analysis,  we  cannot  afford  not  to  do 
the  few  thousand  analyses. 

Randomization  analysis,  as  urged  by  R.  A.  Fisher,  and  founded  by  E.  J.  G.  Pitman  (and 
B.  I,.  Welch)  grew  up  in  an  era  of  expensive  computing.  As  a  result  even  Pitman's  combina- 
torially  obtained  higher  moments  were  forgotten  in  practice.  (No  one  wanted  to  sum  the 
cubes  and  4th  powers  of  the  observed  differences;  summing  squares  was  hard  enough.) 
Moreover,  our  attention  was  focused  on  means,  as  the  summary  to  be  used,  in  large  part 


because  it  was  only  for  them  that  the  necessary  formula  manipulation  seemed  feasible. 

In  the  interim/  while  computing  has  become  so  many  powers  of  ten  cheaper/  we  have 
learned  a  lot  about  when  using  means  vill  give  high  stringency  -  and  when  it  will  not  The 
way  to  provide  both  the  validity  that  randomization  has  always  provided,  and  the  robust 
stringency  that  it  can  provide  is  to  plan  to  actually  do  many  alternative  analyses,  using  what* 
ever  summary  -  often  one  providing  robust  stringency  -  seemed  most  desirable  when  the 
experiment  was  planned!  By  doing  MANY  reassignments  explicitly,  we  can  be  quite  sure  of 
what  we  are  doing. 

For  experiments  of  certain  sizes,  we  can  use  complete  randomization,  which  means 
making  calculations  for  all  possible  assignments  (within  the  general  framework  of  the  expert* 
ment),  but  we  cannot  do  this  for  all  sizes  of  experiments!  2'  possible  assignments  means  2* 
reanalyses,  and  we  can  only  afford  this  for  modest  t  (The  bound  will  go  up  year  after  year). 
So  we  will  have  to  consider  combine  explicit  reassignment  with  limited  randomizations  - 
something  we  can  approach  in  different  ways. 

4.  The  details  of  explicit  reassignment  with  complete  randomization  (2*  case) 

If  we  have  t  blocks,  each  composed  of  2  units,  and  if  treatment  and  control  are  each  to 
appear  once  in  each  block,  there  will  be  2*  possible  assignments. 

If  we  have  chosen  a  summary,  for  simplicity  based  on  the  t  within-block  treatment* 
MINUS*control  differences  (more  general  summaries  cause  no  difficulties),  it  is  straight* 
forward  to  introduce  all  possible  sequences  of  t  ±  signs,  and  thus,  lor  each  such  sequence,  t 
values,  one  per  block,  of 

possible  difference  ■  ±  actual  difference 

whose  2f  summaries  correspond  to  all  possible  2'  reanalyses.  For  each  such  "randomiza¬ 
tion"  there  is  a  value  of  the  corresponding  summary  -  which  might  be  a  mean,  a  midmean, 
some  other  trimmed  mean,  or  a  blweight  of  the  n  possible  differences. 

We  can  take  these  2'  summary  values,  rank  them,  and  then  take,  as  our  P*value: 

P*value  -  rank  ofactual  SSSiSBSSSt 
2 * 

If  we  want  a  one-tailed  P-value,  we  will  rank  from  one  end  ••  the  top  or  the  bottom,  as 
appropriate  -  starting  with  the  relevant  extreme. 

If  we  have  a  two-tailed  P-value,  we  will  rank  from  both  ends  and  combine  the  results 
into  a  single  ranking.  Here 


Rank 

Rank 

Combined 

above 

below 

rank 

1 

2" 

2 

2" 

1 

2 

2 

2M 

4 

2M 

2 

4 

3 

2"*2 

6 

2" -2 

3 

6 

4 

2**3 

8 

2* -3 

4 

8 

(and  so  on) 

is  conventional.  If,  however,  we  want  to  give  the  rank  from  above  a  shade  of  preference,  it 
is  both  legitimate  and  proper  to  use  (instead): 
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Rank 

Rank 

Combined 

abovt 

below 

rank 

1 

V 

1 

2" 

1 

2 

2"*1 

2 

4 

3 

2"  *2 

5 

2" -2 

3 

6 

4 

2" *3 

7 

2* -3 

4 

8 

(and  10  on) 

Both  of  these  wanking*  satisfy  tha  essential  Inequality: 

#  of  assignments  with  combined  rank  £  k  IS  ITSELF  £  k 

The  first  of  these  reranking*  alternate*  <  and  £,  while  the  tecond  always  ha*  £.  The  first  is 
symmetrical,  the  second  is  not  wasteful. 

If  we  proceed  in  this  way,  choosing  our  summary  (and  our  combined  ranking  pro* 
cedure,  if  we  are  to  use  one)  in  advance  of  our  data  gathering,  and  doing  a  trustworthy  ]ob 
of  selecting  one  assignment  out  of  2*  sufficiently  nearly  at  random,  the  validity  of  our  P* 
value  is  guaranteed  (by  the  quality  of  our  randomization,  which  we  know  all  about,  and 
which  others  can  inspect,  if  we  keep  good  records). 

It  may  be  worthwhile  to  explain  how  to  guarantee  that  1  out  of  2f  -  say  1  out  of  1024  - 
will  be  chosen  very,  very  nearly  at  random.  A  simple  approach  calls  for  3  people  who 

*  can  be  trusted  to  work  independently  of  one  another,  AND 

*  of  whom  at  least  one  is  both  honest  and  skilted  In  randomization. 

We  then  proceed  as  follows: 

*  number  all  possible  1024  assignments  from  0  to  1023, 

e  request  each  of  the  three  persons  to  produce  a  random  integer  between  0  and  1023, 
AND 

e  add  up  the  three  integers,  reduce  the  result  modulo  1024,  and  adopt  the  corresponding 
assignment. 

No  two  people  could  combine  to  keep  the  third  from  making  the  overall  choice  ran* 

dom. 

How  well  we  do  about  stringency  (or  other  desideratum)  depends  on  how  well  we 
choose  our  summary.  We  do  know  which  summary  we  chose.  We  may  know  how  this 
summary  performs  for  a  more  or  less  flexible  statistical  model  ••  today  almost  certain  to 
involve  an  infinite  population.  The  h  absolute  differences  we  actually  have  may  collectively 
throw  some  light  on  how  the  summary  performed  in  our  specific  case. 

We  are  just  about  as  well  off  about  stringency  **  about  robustness  of  stringency  -  as  we 
ever  are,  and  we  are  as  well  off  about  validity  as  we  can  be  -  since  we  depend  only  on  a 
process  that  can  be  monitored,  recorded  and  inspected. 

A  picture  may  help  us  to  get  a  feel: 


10 

blocks 


1024  possible  assignments 


We  have  a  itock  of  1024  assignments  and  chose  one  at  random.  We  summarize  sets  of  10 
values  of  the  form  {±  an  observed  difference}  1024  times,  to  get  1024  values  of  a  given  sum* 
mary. 


5.  Some  kinds  of  limited  randomization. 

The  complete-randomiration  design  and  analysis  of  experiment  just  described  is  prob¬ 
ably  most  reasonable  for  t  blocks,  where  9  *  t  *  12.  At  2*  ■  512,  we  still  have  enough 
assignments  so  that  comparison  with  both  5%  and  1%  points  is  not  serioush  plagued  by 
granularity.  At  21J  ■  4096,  we  can  still  face  doing  one  analysis  for  each  of ;  ur-thousand- 
odd  assignments.  But  what  of  more  than  12  blocks  uf  2. 

We  shall  discuss  three  approaches,  briefly. 


*  ttacking  up  * 

We  can  stack  our  n  blocks  into  k  stacks,  locking  together  the  assignment  of  treatment 
and  control  for  blocks  in  the  same  stack.  We  then  need  one  random  ±  sign  per  stack. 

If,  for  example,  we  have  24  blocks,  we  may  stack  them  in  12  stacks,  say  #24  with  #1, 
#23  with  #2,  ...,  #13  with  #12,  and  in  each  stack  we  may  choose  a  locking  together, 
perhaps  like  this 


#1  y  z  #2  y  z  #12  y  c 

#24  z  y  #23  y  z  #13  z  y 

where  we  plan  to  randomly  assign  treatment  and  control  to  y,  and  z  or  to  z,  and  y  in  each 
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stack  separately. 

All  our  techniques,  discussed  today  or  not,  WORK  just  as  WELL  for  stacks  -  locked- 
together  groupings  of  one  or  more  blocks  -  as  for  single  blocks.  So,  as  we  go  on,  we  will 
usually  talk  about  itacke  since  this  covers  all  cases. 

With  k  stacks  stacked  and  locked,  there  are  2k  possible  assignments,  and  we  have  the 
full  validity  that  we  had  before,  and  the  same  preference  for  9  *  k  (as  12).  We  may  have  to 
be  careful  to  use  a  summary  that  is  appropriate  for  blocks  In  stacks.  If  we  do  this  carefully, 
there  is  no  reason  why  the  stacks  should  all  contain  the  same  number  of  blocks! 

Used  alone,  this  la  already  a  very  flexible  form  of  limited  randomization  -  one  that 
seems  particularly  appropriate  for  treatment-control  differences  whose  distribution  seems 
vaguely  Gaussian  In  shape  (because  we  are  likely*  to  add  up  the  differences  within  each 
stack). 

Used  together  with  those  to  be  described  next,  "stacking  up”  contributes  to  an  even 
greater  degree  of  flexibility. 

*  orthogonal  arrays  • 

There  are  a  wide  variety  of  ways  to  use  orthogonal  arrays  -  1  am  about  to  tell  you  of  a 
rather  new  one.  (A  convenient  reference  is  Raktoe,  Hedayat,  and  Federer  1981,  Factorial 
Designs, ,  specifically  pages  168-189  and  certain  of  the  references  at  pages  168-193.) 

A  simple  example  is 

1  0  0  0  0  1  1  1 

0  10  0  10  11 

0  0  10  110  1 

0  0  0  1  1  1  1  0 

whose  main  virtue  is  that,  if  we  pick  out  any  three  rows,  ray  the  first,  third,  and  fourth 

1  0  0  0  0  1  1  1 

0  0  10  110  1 

0  0  0  1  1  1  1  0 

each  of  the  2s  possible  columns  appears  an  equal  number  of  timea  -  namely,  here,  one. 

Officially,  this  is  an  orthogonal  array  of  size,  n»8;  t«4  constraints;  s“2  levels  (or  alpha¬ 
bet  size  2);  and  strength  d«3  (meaning  that  any  three  rows  arc  completely  balanced. 

In  conventional  notation  it  is  an 

OA(6,  4,  2,  3) 

Mote  generally,  an  orthogonal  array  is  an 

OA(n,  t,  s,  d) 

or  an 

OA(n,  t,  s,  d,  X) 

where  X  is  the  common  number  of  appearances  of  all  subcolumns  for  any  chosen  set  of  d 
rows.  Our  example  is  thus  also  an  OA(8,  4,  2,  3;  1). 

Lest  you  think  that  every  OA  is  a  fractional  factorial,  let  us  look  at  an  OA  (18,  7,  3,  2), 
namely: 


•But  do  not  tuve  to! 
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000111222000111222 

012012012012012012 

012120201201012120 

021102210102210021 

012120120012201201 

012201012120201120 

012012201120120201 


For  each  of  the  (gj  ■  21  choice*  of  two  row*  from  this  array,  we  will  find 

0  0  0  1  1  1  2  2  2 
0  1  2  0  1  2  0  1  2 

each  occurring  twice.  Hence  the  orthogonality  of  itrtngth  two. 

The  "classical"  employment  of  orthogonal  arrays  is  at  follows: 


0  t 

factors 


n  plot*  or  nini 


We  are  going  to  use  them  quite  differently,  namely: 


a  potential  | 


aMigiunente 
random!  aationi 


r 


on*  obtarvtd 
difference 
in  each  row 


n  potential  rummarlei 


A  rather  extreme  example  -  one  that  will  probably  prove  useful  now  and  then  -  is 
shown  below: 
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*  W  . 4 ►  ’  s"*V  ‘  •.  V*V«V'V'Y»VA  "  ** 


the  actual  axpartmant 

(Also  bear  in  mind  that  taking  all  2*  assignments  as  potential  assignments  -  complete 
randomization  for  t  stacks  -  is  also  using  an  OAI)  Each  of  our  assignments,  then  specifies, 
for  each  of  k  stacks  (each  stack  having  one  or  more  blocks,  how  the  treatment  and  control 
are  to  be  assigned  to  the  halves  of  that  stack  -  and  hence  to  the  units  of  each  of  its  blocks. 

The  duign  of  the  experiment  can  usefully  be  thought  of  as  including  the  orthogonal 
array  from  which  the  actual  assignment  is  to  be  drawn  at  random. 

But  where  do  we  get  such  medium-sized  orthogonal  arrays? 

When  this  question  first  arose,  I  was  fortunate  enough  to  ask  Neil  Sloane,  who  reached 
in  his  supply  kit  and  produced  a  small  array  of  such  OA's,  each  one  corresponding  to  a 
fairly  well-known  code  (in  the  sense  of  signal-coding  theory)  These  included  (the 
parentheses  give  their  labels  as  fractional  factorial  designs): 


OA(  512,  23,  2,  4) 
Oa(2048,  23,  2,  6) 
OA(1024,  24,  2,  5) 
OA(1024,  32,  2,  4) 
OA(2048,  33,  2,  5) 
OA(2048,  63,  2,  4) 
OA(4096,  64,  2,  5) 


(2°’w) 

(2«-») 

(224-U) 

(2m-«) 

(2“-“) 
(2 :«-”) 


where  we  recall  that,  for  our  uses,  the  notations  mean  OA(#  of  assignments,  up  to  this 
many  blocks,  2,  strength).  Clearly  we  do  quite  well  up  to  60  blocks  (or  even  60  stacks  of 
blocks), 

A  matter  of  convenience  and  ease  of  use  arises  because 
a)  these  arrays  are  coscts  of  certain  codes, 


b)  if  one  coset  is  ait  OA,  so  is  any  other  coset  of  the  same  code,  with  the  same  parame¬ 
ters, 

c)  no  two  coaets  overlap,  while  taken  together  they  indude  all  2*  assignments. 

As  a  result,  we  can  proceed  as  follows: 

1)  choose  a  code,  and  thus  a  family  of  cosets, 

2)  choose,  randomly,  any  assignment  as  the  actual  assignment  -  and  choose  the  coset  that 
indudes  this  assignment  as  the  set  of  possible  assignments. 

This  convenient,  one-step  process  is  logically  and  philosophically  equivalent  to  the 
more  obviously  sound  two-step  process: 

(i)  choose  a  coset  at  random, 

(ii)  choose  an  assignment  in  the  eoset  at  random 

Besides  convenience,  and  making  all  assignments  candidates  for  possible  assignments, 
this  use  of  OA's  has  a  particular  advantage  in  what  we  might  describe  as  ANTI-ROBUST 
situations,  where  there  are  likely  to  be  a  few  very  large  differences  -  for  a  few  stacks,  or  for 
a  few  blocks  -  whose  effects  we  want  to  study  and  focus  upon,  rather  than  to  set  aside  -  as 
the  use  of  a  robust  summary  would  do.  Mot<t  weather-modification  experiments,  for 
instance,  whether  concerned  with  rainfall  increase  or  hail-damage  reduction,  require  antlro- 
bust  analysis,  since  the  occasional  large  storms  yield  most  of  the  water  and  the  occasional 
violent  storms  cause  most  of  the  damage. 


*  Gautaian  combination  of  aubexperimente  * 

The  third  approach  to  many  blocks  need  not  take  us  long.  If  we  are  willing  to 

•  divide  the  experiment  into  subexperiments 

•  summarize  these  subexperiments  separately,  and  then 

•  combine  these  summaries 

we  have  only  to  convert  the  summary  of  each  subexperiment  into  a  reasonable  (discrete)  fac¬ 
simile  of  a  unit  Gaussian  deviate,  add  up  the  facsimiles,  and  divide  by  (#  of  facsimiles)1*, 
referring  the  result  to  the  unit  Gaussian  distribution. 

So  how  are  we  to  get  our  nearly  unit  Gaussians  from  our  subexperiment  summaries? 
We  may  as  well  plan  to  begin  by  ranking  our  21  possible  summaries  for  each  subexperiment. 

A  little  calculation  of  cumulants  then  leads  us  to  suggest  taking 


unit  Gautaian  aeon 


Gan 


-l 


1- 


3f-2 
3m -1 


where 


i  ■  given  rank,  m  ■  2* 


and 


Gau{ )  it  the  unit  Gautaian  cumulative. 

This  seems  fairly  certain  to  work  well  for  3  or  more  subexperiments  with  2i  fe  236.  (The 
case  of  2  subexperiments  can  be  bypassed  treated  directly,  without  Gaussianizing,  by  a  dif¬ 
ferent  calculation.) 


*  combination  * 

We  can  clearly  use  these  devices  of  combination  in  sequence  as  well  as  alone. 

*  protective  limitatione  * 


487 


There  have  been  recurrent  suggestions  for  limiting  randomization  to  avoid  unusual 
assignments  —  principally  very  unbalanced  assignments,  occasionally  too  regular  ones, 
While  this  is  clearly  more  important  for  unblocked  experiments  than  for  blocked  ones,  there 
is  no  reason  for  not  superposing  such  deletions  on  any  of  the  randomization-reassignment 
patterns,  complete  or  limited,  we  have  contidered  so  long  as:  so  long  as,  before  choosing  the 
actual  assignment: 

•  we  decide  on  which  deletions  are  to  be  made,  AND 

•  we  implement  these  deletions  UNIFORMLY,  to  potential  and  actual  assignments  alike. 

6.  Compromising  by  rank  combination. 

All  procedures  that  are  robust  of  stringency  are  compromises,  at  least  implicitly!  When 
we  want  the  greatest  care,  we  are  likely  to  want  an  especially  thorough  compromise  as  our 
chosen  summary.  If  we  can  find  a  sufficiently  broadly  oriented  compromise  summary,  so 
much  the  better.  If  not,  then  we  may  need  to  start  with  two  or  more  (compromise)  sum¬ 
maries,  and,  further,  compromise  them  with  one  another  explicitly. 

How  should  we  do  this?  To  make  separate  significance  tests  with  each,  and  then  dou¬ 
ble  the  smaller  of  the  two  P-values  is  the  counsel  of  the  Bonferroni  inequality.  This  would 
almost  surely  be  wasteful  -  probably  substantially  so.  So  what  ia  surely  better? 

•  To  rank  the  2*  assignments  according  to  each  summary  separately  -  and  then  to  com¬ 
bine  the  two  sets  of  ranks  in  some  appropriate  way. 

A  simple  and  natural  way  is  to  put  the  two  ranks  for  each  assignment  in  lexicographi¬ 
cal  order  within  each  pair,  and  then  sort  the  ordered  pairs  in  lexicographical  order  of  pairs, 
and  use  this  latter  order  to  define  a  combined  rank.  The  resulting  correspondence 


observed  numbers 
possible  assignment 


combined  rank 


does  not  at  all  depend  on  which  of  the  pouible  assignments  was  the  actual  assignment.  So 
we  know  that 

combined  rank 
2* 

is  a  legitimate  P-value,  so  that  we  are  making  a  proper  significance  test.  If  the  two  sum¬ 
maries  were  secretly  the  same,  so  too  would  be  the  individual  ranks  -  and  the  combined 
ranks  -  so  that  the  compromise  P-value  would  reduce  to  the  common  P-value  for  the  indivi¬ 
dual  summaries.  If  the  two  summaries  are  very,  very  different,  the  result  of  this  sort  of  rank 
combination  would  be  very  dose  to  the  Bonferroni  result. 

A  hypothetical  example  may  help  to  fix  the  ideas. 


alignment 

Ranks  according  to 

ordered 

combin 

# 

1st  summary 

2nd  summary 

pair 

rank 

351 

5 

1 

(1,5) 

1 

679 

1 

29 

(1,29) 

2 

224 

2 

42 

(2,42) 

3 
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97 

2 

(2,97) 

4 

976 

3 

4 

0,4) 

5 

214 

14 

3 

(3,14) 

6 

723 

4 

53 

(4,53) 

7 

431 

8 

5 

(5,8) 

8 

92 

t, 

6 

(6,6) 

9 

127 

7 

8 

(7,8) 

10 

923 

11 

7 

(7,11) 

11 

23 

9 

11 

(9,11) 

12 

Notice  that,  except  for  tie-breaking,  the  firat  7  line*  follow  the  Bonforroni  bound  to 
within  a  unit,  but,  that  beyond  7,  the  similarities  in  the  two  separate-summary  rankings  are 
so  great  as  to  produce  deviations  from  the  bound.  Indeed,  far  (S,x),  (6,y),  (7,z)  and  (8,w) 
together  we  use  up  only  4  ranks  instead  of  8. 

This  sort  of  indirect  explicit  compromise  is  certainly  interesting,  and  probably  promis¬ 
ing. 

7.  Introduction  to  3  or  more  treatments. 

What  if  we  do  3  treatments  in  n  blocks  of  3?  How  do  we  proceed? 

Unless  the  treatments  are  associated  with  the  "levels”  of  a  quantitative  x,  our  most 
likely  purpose  is  to  assess  the  significance  of  (to  say  how  sure  we  seam  to  be  of  the  direction 
of)  the  pair-wise  comparisons.  So  how  should  we  compare  any  two  treatments? 

Again  there  is  more  than  one  approach. 

Let  the  three  treatments  be  F,  G,  H.  there  are  6*3!  ways  of  assigning  one  each  to  the 
3  units  of  any  block.  With  n  blocks;  and  all  assignments  possible,  there  are  6*  possible 
assignments. 

If  we  want  to  compare  F  with  G,  and  do  this  without  interference  from  treatment  H, 
which  may  produce  very  different  values,  a  very  simple  thing  is  to  only  consider  those  2" 
assignments  as  "possible"  in  which  the  UNordertJ  pair  F,  G  are  assigned  to  the  same  two 
units  in  each  block. 

This  means  that  we  look  at  one  family  of  2*  assignments  to  compare  F  with  G  and 
another  family  of  2M  to  compare  F  with  H.  These  two  families  will,  in  fact,  have  only  one 
assignment  in  common  -  namely  the  actual  one.  (This  means  that  we  do  not  have  alterna¬ 
tive  rankings  for  different  pah-wise  comparisons,  so  that  we  seem  to  be  driven  back  on  a 
Bonferronl-eombined  P-value.) 

Like  the  cosets  in  our  code-derived  OA's,  the  families  of  2*  •*  out  of  6*  -  assignments 
are  non-overlapping.  The  selection  of  any  one  of  the  2*  members  of  such  a  family  as  the 
actual  assignment  forces  that  family  on  us  as  the  basts  for  a  particular  paired  comparison. 
Although  there  is  not,  In  this  case,  a  clear  two-step  equivalence,  the  one-step  procedure  is 
just  as  valid  as  the  one-step  procedure  in  the  earlier  Instance. 

*  iteond  approach  * 

To  avoid  both 

i)  the  potential  effects  of  the  other  treatment(s)  —  regarded  as  well-behaved  though  possi¬ 
bly  large,  and 

ii)  the  full  Bonferronl  penalty,  in  those  circumstances  where  it  is  not  all  justified, 


we  have  to  be  somewhat  trickier  -  though  the  resulting  procedure  is  almost  surely  nearly 
enough  legitimate  and  valid.  (Any  doubts  that  we  may  have  of  practical  performance  must 
then  be  devoted  to  its  stringency.) 

What  appears  to  be  the  simplest  such  approach  is  the  following: 

•  use  only  a  cyclic  three  of  the  six  assignments  to  the  units  of  any  block  -  e.g.  use  FGH, 
HFG  or  GHF  but  not  HGF,  FGH,  or  GFH  (or  £&  perse) 

•  bend  the  rules  by  analyzing  the  actual  two-way  table 


additlvely  -  perhaps  by  lomedian  polish,  and 
•  adjust  every  H-actual  value  by  subtracting 


H-fit  MINUS 


from  it 

•  summarize  the  comparisons  of  F-possible  and  G-possible  for  all  3"  possible  assign¬ 
ments,  using  whichever  two  of  F-actual,  G-actual,  and  H-  adjusted  (notice  that  only  one 
of  the  three  is  adjusted)  correspond  to  the  possible  assignment  at  each  of  the  blocks  in 
question. 

e  repeat  the  last  two  steps  with  first  F,  and  then  G,  replacing  H. 

The  result  of  all  this  is  a  ranking  for  each  combination  of  one  of  the  3  ■  (jjj)  paired 

comparisons  with  each  of  the  possible  assignments.  We  can  go  on  to  combine  the  3  ranks 
for  a  given  possible  assignments  in  each  of  a  number  of  ways,  leading  to  P-values  for  a 
number  of  kinds  of  multiple  comparisons  -  ranging  from  multiple  comparisons  focusing  on 
finding  at  least  one  difference  significant  to  multiple  comparisons  focusing  on  finding  as 
many  differences  as  possible  significant, 


8.  Cosing  comment. 

Not  only  does  limited  randomization  with  explicit  reassignment  provide  us  with 
guaranteed  validity,  but,  as  the  previous  sentence  about  multiple  comparisons  suggest,  it 
provides  us  with  a  basis  for  better  understanding  of  a  variety  of  statistical  devices. 
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